2023-04-09 07:53:30,130 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-04-09 07:53:30,408 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934 2023-04-09 07:53:30,421 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-04-09 07:53:30,421 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801, deleteOnExit=true 2023-04-09 07:53:30,422 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-04-09 07:53:30,422 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/test.cache.data in system properties and HBase conf 2023-04-09 07:53:30,422 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/hadoop.tmp.dir in system properties and HBase conf 2023-04-09 07:53:30,423 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/hadoop.log.dir in system properties and HBase conf 2023-04-09 07:53:30,423 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/mapreduce.cluster.local.dir in system properties and HBase conf 2023-04-09 07:53:30,424 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-04-09 07:53:30,424 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-04-09 07:53:30,541 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-04-09 07:53:30,977 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-04-09 07:53:30,981 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-04-09 07:53:30,982 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-04-09 07:53:30,982 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-04-09 07:53:30,982 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-09 07:53:30,982 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-04-09 07:53:30,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-04-09 07:53:30,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-09 07:53:30,983 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-09 07:53:30,984 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-04-09 07:53:30,984 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/nfs.dump.dir in system properties and HBase conf 2023-04-09 07:53:30,985 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/java.io.tmpdir in system properties and HBase conf 2023-04-09 07:53:30,985 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-09 07:53:30,985 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-04-09 07:53:30,985 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-04-09 07:53:31,578 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-09 07:53:31,583 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-09 07:53:31,896 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-04-09 07:53:32,082 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-04-09 07:53:32,101 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-09 07:53:32,146 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-04-09 07:53:32,177 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/java.io.tmpdir/Jetty_localhost_45397_hdfs____bpbjtm/webapp 2023-04-09 07:53:32,321 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45397 2023-04-09 07:53:32,367 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-09 07:53:32,367 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-09 07:53:32,873 WARN [Listener at localhost/40795] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-09 07:53:32,989 WARN [Listener at localhost/40795] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-09 07:53:33,013 WARN [Listener at localhost/40795] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-09 07:53:33,020 INFO [Listener at localhost/40795] log.Slf4jLog(67): jetty-6.1.26 2023-04-09 07:53:33,027 INFO [Listener at localhost/40795] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/java.io.tmpdir/Jetty_localhost_41561_datanode____3b2tfe/webapp 2023-04-09 07:53:33,130 INFO [Listener at localhost/40795] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41561 2023-04-09 07:53:33,513 WARN [Listener at localhost/35977] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-09 07:53:34,014 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdea589536b9c3357: Processing first storage report for DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb from datanode 1d6642eb-77e4-4c9a-88ef-513280ce7b16 2023-04-09 07:53:34,016 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdea589536b9c3357: from storage DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb node DatanodeRegistration(127.0.0.1:43737, datanodeUuid=1d6642eb-77e4-4c9a-88ef-513280ce7b16, infoPort=45493, infoSecurePort=0, ipcPort=35977, storageInfo=lv=-57;cid=testClusterID;nsid=779894858;c=1681026811671), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-04-09 07:53:34,016 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xdea589536b9c3357: Processing first storage report for DS-32b7720e-fd97-44cb-b220-82d6bab32e3c from datanode 1d6642eb-77e4-4c9a-88ef-513280ce7b16 2023-04-09 07:53:34,017 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xdea589536b9c3357: from storage DS-32b7720e-fd97-44cb-b220-82d6bab32e3c node DatanodeRegistration(127.0.0.1:43737, datanodeUuid=1d6642eb-77e4-4c9a-88ef-513280ce7b16, infoPort=45493, infoSecurePort=0, ipcPort=35977, storageInfo=lv=-57;cid=testClusterID;nsid=779894858;c=1681026811671), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-09 07:53:34,097 DEBUG [Listener at localhost/35977] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934 2023-04-09 07:53:34,176 INFO [Listener at localhost/35977] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/zookeeper_0, clientPort=57867, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-04-09 07:53:34,188 INFO [Listener at localhost/35977] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=57867 2023-04-09 07:53:34,199 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:34,202 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:34,850 INFO [Listener at localhost/35977] util.FSUtils(471): Created version file at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c with version=8 2023-04-09 07:53:34,850 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/hbase-staging 2023-04-09 07:53:35,168 INFO [Listener at localhost/35977] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-04-09 07:53:35,676 INFO [Listener at localhost/35977] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-04-09 07:53:35,717 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:35,718 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:35,718 INFO [Listener at localhost/35977] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-09 07:53:35,718 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:35,718 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-09 07:53:35,864 INFO [Listener at localhost/35977] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-04-09 07:53:35,943 DEBUG [Listener at localhost/35977] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-04-09 07:53:36,044 INFO [Listener at localhost/35977] ipc.NettyRpcServer(120): Bind to /172.31.14.131:44231 2023-04-09 07:53:36,054 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:36,056 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:36,080 INFO [Listener at localhost/35977] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44231 connecting to ZooKeeper ensemble=127.0.0.1:57867 2023-04-09 07:53:36,126 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:442310x0, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-09 07:53:36,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:44231-0x10167e97a960000 connected 2023-04-09 07:53:36,156 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-09 07:53:36,157 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-09 07:53:36,160 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-09 07:53:36,168 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44231 2023-04-09 07:53:36,169 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44231 2023-04-09 07:53:36,169 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44231 2023-04-09 07:53:36,170 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44231 2023-04-09 07:53:36,170 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44231 2023-04-09 07:53:36,176 INFO [Listener at localhost/35977] master.HMaster(444): hbase.rootdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c, hbase.cluster.distributed=false 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-09 07:53:36,246 INFO [Listener at localhost/35977] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-09 07:53:36,251 INFO [Listener at localhost/35977] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-09 07:53:36,254 INFO [Listener at localhost/35977] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34907 2023-04-09 07:53:36,256 INFO [Listener at localhost/35977] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-09 07:53:36,262 DEBUG [Listener at localhost/35977] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-09 07:53:36,263 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:36,265 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:36,267 INFO [Listener at localhost/35977] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34907 connecting to ZooKeeper ensemble=127.0.0.1:57867 2023-04-09 07:53:36,271 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:349070x0, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-09 07:53:36,272 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34907-0x10167e97a960001 connected 2023-04-09 07:53:36,272 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-09 07:53:36,273 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-09 07:53:36,274 DEBUG [Listener at localhost/35977] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-09 07:53:36,274 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34907 2023-04-09 07:53:36,275 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34907 2023-04-09 07:53:36,275 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34907 2023-04-09 07:53:36,276 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34907 2023-04-09 07:53:36,276 DEBUG [Listener at localhost/35977] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34907 2023-04-09 07:53:36,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:36,286 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-09 07:53:36,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:36,306 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-09 07:53:36,306 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-09 07:53:36,306 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:36,307 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-09 07:53:36,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,44231,1681026814996 from backup master directory 2023-04-09 07:53:36,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-09 07:53:36,311 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:36,311 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-09 07:53:36,312 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-09 07:53:36,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:36,315 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-04-09 07:53:36,316 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-04-09 07:53:36,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/hbase.id with ID: 43a30981-c5e4-4695-90fa-66bc967fd770 2023-04-09 07:53:36,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:36,451 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:36,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x3ac6a0e1 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:36,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@92d05fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:36,543 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-09 07:53:36,545 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-04-09 07:53:36,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-04-09 07:53:36,566 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-04-09 07:53:36,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-09 07:53:36,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-09 07:53:36,574 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-09 07:53:36,607 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store-tmp 2023-04-09 07:53:36,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:36,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-09 07:53:36,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:53:36,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:53:36,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-09 07:53:36,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:53:36,635 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:53:36,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-09 07:53:36,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/WALs/jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:36,661 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C44231%2C1681026814996, suffix=, logDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/WALs/jenkins-hbase4.apache.org,44231,1681026814996, archiveDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/oldWALs, maxLogs=10 2023-04-09 07:53:36,727 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK] 2023-04-09 07:53:36,737 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-04-09 07:53:36,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/WALs/jenkins-hbase4.apache.org,44231,1681026814996/jenkins-hbase4.apache.org%2C44231%2C1681026814996.1681026816672 2023-04-09 07:53:36,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK]] 2023-04-09 07:53:36,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:36,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:36,807 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,809 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,871 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,877 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-04-09 07:53:36,901 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-04-09 07:53:36,912 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:36,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,921 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-04-09 07:53:36,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:53:36,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66165351, jitterRate=-0.014059439301490784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:53:36,942 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-09 07:53:36,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-04-09 07:53:36,961 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-04-09 07:53:36,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-04-09 07:53:36,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-09 07:53:36,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-09 07:53:36,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-04-09 07:53:36,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-04-09 07:53:37,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-04-09 07:53:37,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-04-09 07:53:37,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-04-09 07:53:37,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-04-09 07:53:37,056 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-04-09 07:53:37,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-04-09 07:53:37,065 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-04-09 07:53:37,067 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:37,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-04-09 07:53:37,069 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-04-09 07:53:37,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-04-09 07:53:37,084 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-09 07:53:37,084 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-09 07:53:37,084 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:37,085 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,44231,1681026814996, sessionid=0x10167e97a960000, setting cluster-up flag (Was=false) 2023-04-09 07:53:37,100 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:37,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-04-09 07:53:37,107 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:37,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:37,120 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-04-09 07:53:37,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:37,125 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp 2023-04-09 07:53:37,180 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(952): ClusterId : 43a30981-c5e4-4695-90fa-66bc967fd770 2023-04-09 07:53:37,186 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-09 07:53:37,192 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-09 07:53:37,192 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-09 07:53:37,196 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-09 07:53:37,197 DEBUG [RS:0;jenkins-hbase4:34907] zookeeper.ReadOnlyZKClient(139): Connect 0x3b2e629f to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:37,202 DEBUG [RS:0;jenkins-hbase4:34907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35e98b8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:37,203 DEBUG [RS:0;jenkins-hbase4:34907] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@31d609ef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-04-09 07:53:37,236 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34907 2023-04-09 07:53:37,241 INFO [RS:0;jenkins-hbase4:34907] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-09 07:53:37,241 INFO [RS:0;jenkins-hbase4:34907] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-09 07:53:37,242 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1023): About to register with Master. 2023-04-09 07:53:37,245 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,44231,1681026814996 with isa=jenkins-hbase4.apache.org/172.31.14.131:34907, startcode=1681026816245 2023-04-09 07:53:37,246 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-04-09 07:53:37,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1681026847260 2023-04-09 07:53:37,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-04-09 07:53:37,267 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-04-09 07:53:37,267 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-04-09 07:53:37,269 DEBUG [RS:0;jenkins-hbase4:34907] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-09 07:53:37,275 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-09 07:53:37,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-04-09 07:53:37,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-04-09 07:53:37,283 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-04-09 07:53:37,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-04-09 07:53:37,284 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-04-09 07:53:37,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,288 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-04-09 07:53:37,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-04-09 07:53:37,290 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-04-09 07:53:37,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-04-09 07:53:37,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-04-09 07:53:37,304 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1681026817299,5,FailOnTimeoutGroup] 2023-04-09 07:53:37,305 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1681026817305,5,FailOnTimeoutGroup] 2023-04-09 07:53:37,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-04-09 07:53:37,307 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,307 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,334 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40491, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-04-09 07:53:37,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,369 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c 2023-04-09 07:53:37,369 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:40795 2023-04-09 07:53:37,369 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-09 07:53:37,376 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-09 07:53:37,377 DEBUG [RS:0;jenkins-hbase4:34907] zookeeper.ZKUtil(162): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,377 WARN [RS:0;jenkins-hbase4:34907] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-09 07:53:37,378 INFO [RS:0;jenkins-hbase4:34907] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-09 07:53:37,378 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1947): logDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/WALs/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,380 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34907,1681026816245] 2023-04-09 07:53:37,389 DEBUG [RS:0;jenkins-hbase4:34907] zookeeper.ZKUtil(162): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,402 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-09 07:53:37,411 INFO [RS:0;jenkins-hbase4:34907] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-09 07:53:37,430 INFO [RS:0;jenkins-hbase4:34907] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-09 07:53:37,434 INFO [RS:0;jenkins-hbase4:34907] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:53:37,434 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,436 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-09 07:53:37,444 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,444 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,444 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,445 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,446 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,446 DEBUG [RS:0;jenkins-hbase4:34907] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-04-09 07:53:37,448 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,448 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,448 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,465 INFO [RS:0;jenkins-hbase4:34907] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-09 07:53:37,467 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34907,1681026816245-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:37,484 INFO [RS:0;jenkins-hbase4:34907] regionserver.Replication(203): jenkins-hbase4.apache.org,34907,1681026816245 started 2023-04-09 07:53:37,484 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34907,1681026816245, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34907, sessionid=0x10167e97a960001 2023-04-09 07:53:37,485 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-09 07:53:37,485 DEBUG [RS:0;jenkins-hbase4:34907] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,485 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34907,1681026816245' 2023-04-09 07:53:37,485 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:37,486 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:37,487 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-09 07:53:37,487 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-09 07:53:37,487 DEBUG [RS:0;jenkins-hbase4:34907] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:37,487 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34907,1681026816245' 2023-04-09 07:53:37,487 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-09 07:53:37,488 DEBUG [RS:0;jenkins-hbase4:34907] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-09 07:53:37,488 DEBUG [RS:0;jenkins-hbase4:34907] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-09 07:53:37,489 INFO [RS:0;jenkins-hbase4:34907] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-09 07:53:37,489 INFO [RS:0;jenkins-hbase4:34907] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-09 07:53:37,600 INFO [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34907%2C1681026816245, suffix=, logDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/WALs/jenkins-hbase4.apache.org,34907,1681026816245, archiveDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/oldWALs, maxLogs=32 2023-04-09 07:53:37,619 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK] 2023-04-09 07:53:37,624 INFO [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/WALs/jenkins-hbase4.apache.org,34907,1681026816245/jenkins-hbase4.apache.org%2C34907%2C1681026816245.1681026817603 2023-04-09 07:53:37,625 DEBUG [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK]] 2023-04-09 07:53:37,718 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:37,720 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:37,720 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c 2023-04-09 07:53:38,137 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:38,140 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-09 07:53:38,143 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/info 2023-04-09 07:53:38,144 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-09 07:53:38,145 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,145 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-09 07:53:38,147 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/rep_barrier 2023-04-09 07:53:38,148 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-09 07:53:38,149 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,149 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-09 07:53:38,151 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/table 2023-04-09 07:53:38,152 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-09 07:53:38,152 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,154 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740 2023-04-09 07:53:38,155 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740 2023-04-09 07:53:38,159 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-09 07:53:38,161 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-09 07:53:38,164 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:53:38,165 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64654803, jitterRate=-0.03656835854053497}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-09 07:53:38,165 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-09 07:53:38,165 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-09 07:53:38,165 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-09 07:53:38,165 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-09 07:53:38,166 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-09 07:53:38,166 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-09 07:53:38,167 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-09 07:53:38,167 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-09 07:53:38,172 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-04-09 07:53:38,172 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-04-09 07:53:38,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-04-09 07:53:38,192 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-04-09 07:53:38,195 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-09 07:53:38,346 DEBUG [jenkins-hbase4:44231] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-04-09 07:53:38,350 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34907,1681026816245, state=OPENING 2023-04-09 07:53:38,359 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-04-09 07:53:38,361 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:38,361 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-09 07:53:38,365 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:38,548 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:38,550 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-09 07:53:38,554 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42110, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-09 07:53:38,565 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-04-09 07:53:38,565 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-09 07:53:38,569 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34907%2C1681026816245.meta, suffix=.meta, logDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/WALs/jenkins-hbase4.apache.org,34907,1681026816245, archiveDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/oldWALs, maxLogs=32 2023-04-09 07:53:38,586 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK] 2023-04-09 07:53:38,591 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/WALs/jenkins-hbase4.apache.org,34907,1681026816245/jenkins-hbase4.apache.org%2C34907%2C1681026816245.meta.1681026818570.meta 2023-04-09 07:53:38,592 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43737,DS-ed24a1d7-a1b9-449c-81e7-24051a83f5eb,DISK]] 2023-04-09 07:53:38,592 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:38,594 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-04-09 07:53:38,613 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-04-09 07:53:38,619 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-04-09 07:53:38,624 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-04-09 07:53:38,624 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:38,624 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-04-09 07:53:38,625 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-04-09 07:53:38,627 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-09 07:53:38,629 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/info 2023-04-09 07:53:38,629 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/info 2023-04-09 07:53:38,630 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-09 07:53:38,631 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,631 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-09 07:53:38,632 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/rep_barrier 2023-04-09 07:53:38,633 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/rep_barrier 2023-04-09 07:53:38,633 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-09 07:53:38,634 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,634 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-09 07:53:38,635 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/table 2023-04-09 07:53:38,635 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/table 2023-04-09 07:53:38,636 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-09 07:53:38,637 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:38,638 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740 2023-04-09 07:53:38,641 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740 2023-04-09 07:53:38,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-09 07:53:38,647 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-09 07:53:38,649 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65872542, jitterRate=-0.018422633409500122}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-09 07:53:38,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-09 07:53:38,658 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1681026818541 2023-04-09 07:53:38,675 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-04-09 07:53:38,676 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-04-09 07:53:38,676 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34907,1681026816245, state=OPEN 2023-04-09 07:53:38,679 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-04-09 07:53:38,679 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-09 07:53:38,683 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-04-09 07:53:38,683 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34907,1681026816245 in 314 msec 2023-04-09 07:53:38,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-04-09 07:53:38,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 504 msec 2023-04-09 07:53:38,693 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5220 sec 2023-04-09 07:53:38,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1681026818693, completionTime=-1 2023-04-09 07:53:38,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-04-09 07:53:38,694 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-04-09 07:53:38,755 DEBUG [hconnection-0x1c2320a1-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:38,758 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42114, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:38,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-04-09 07:53:38,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1681026878776 2023-04-09 07:53:38,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1681026938776 2023-04-09 07:53:38,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 81 msec 2023-04-09 07:53:38,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44231,1681026814996-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:38,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44231,1681026814996-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:38,806 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44231,1681026814996-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:38,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:44231, period=300000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:38,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-04-09 07:53:38,815 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-04-09 07:53:38,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-04-09 07:53:38,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-09 07:53:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-04-09 07:53:38,839 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:53:38,843 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:53:38,869 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:38,871 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c empty. 2023-04-09 07:53:38,872 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:38,872 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-04-09 07:53:38,901 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:38,904 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0b0f4838135d08da8ef165e510cc0c2c, NAME => 'hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 0b0f4838135d08da8ef165e510cc0c2c, disabling compactions & flushes 2023-04-09 07:53:39,319 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. after waiting 0 ms 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,319 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,319 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 0b0f4838135d08da8ef165e510cc0c2c: 2023-04-09 07:53:39,324 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:53:39,340 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1681026819327"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026819327"}]},"ts":"1681026819327"} 2023-04-09 07:53:39,366 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:53:39,368 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:53:39,372 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026819368"}]},"ts":"1681026819368"} 2023-04-09 07:53:39,377 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-04-09 07:53:39,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0b0f4838135d08da8ef165e510cc0c2c, ASSIGN}] 2023-04-09 07:53:39,390 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0b0f4838135d08da8ef165e510cc0c2c, ASSIGN 2023-04-09 07:53:39,391 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=0b0f4838135d08da8ef165e510cc0c2c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:53:39,542 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0b0f4838135d08da8ef165e510cc0c2c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:39,543 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1681026819542"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026819542"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026819542"}]},"ts":"1681026819542"} 2023-04-09 07:53:39,547 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 0b0f4838135d08da8ef165e510cc0c2c, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:39,708 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,708 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0b0f4838135d08da8ef165e510cc0c2c, NAME => 'hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:39,709 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,709 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:39,709 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,709 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,711 INFO [StoreOpener-0b0f4838135d08da8ef165e510cc0c2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,713 DEBUG [StoreOpener-0b0f4838135d08da8ef165e510cc0c2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/info 2023-04-09 07:53:39,713 DEBUG [StoreOpener-0b0f4838135d08da8ef165e510cc0c2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/info 2023-04-09 07:53:39,714 INFO [StoreOpener-0b0f4838135d08da8ef165e510cc0c2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0b0f4838135d08da8ef165e510cc0c2c columnFamilyName info 2023-04-09 07:53:39,714 INFO [StoreOpener-0b0f4838135d08da8ef165e510cc0c2c-1] regionserver.HStore(310): Store=0b0f4838135d08da8ef165e510cc0c2c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:39,716 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,717 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,722 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:53:39,725 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:53:39,726 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0b0f4838135d08da8ef165e510cc0c2c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68370665, jitterRate=0.018802300095558167}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:53:39,726 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0b0f4838135d08da8ef165e510cc0c2c: 2023-04-09 07:53:39,728 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c., pid=6, masterSystemTime=1681026819701 2023-04-09 07:53:39,731 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,731 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:53:39,733 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0b0f4838135d08da8ef165e510cc0c2c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:39,733 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1681026819732"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026819732"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026819732"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026819732"}]},"ts":"1681026819732"} 2023-04-09 07:53:39,742 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-04-09 07:53:39,742 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 0b0f4838135d08da8ef165e510cc0c2c, server=jenkins-hbase4.apache.org,34907,1681026816245 in 191 msec 2023-04-09 07:53:39,746 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-04-09 07:53:39,746 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=0b0f4838135d08da8ef165e510cc0c2c, ASSIGN in 355 msec 2023-04-09 07:53:39,747 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:53:39,748 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026819747"}]},"ts":"1681026819747"} 2023-04-09 07:53:39,750 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-04-09 07:53:39,753 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:53:39,756 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 926 msec 2023-04-09 07:53:39,839 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-04-09 07:53:39,841 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-04-09 07:53:39,841 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:53:39,875 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-04-09 07:53:39,891 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-09 07:53:39,898 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-04-09 07:53:39,908 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-04-09 07:53:39,918 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-09 07:53:39,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-04-09 07:53:39,934 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-04-09 07:53:39,937 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-04-09 07:53:39,937 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.625sec 2023-04-09 07:53:39,939 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-04-09 07:53:39,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-04-09 07:53:39,941 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-04-09 07:53:39,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44231,1681026814996-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-04-09 07:53:39,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,44231,1681026814996-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-04-09 07:53:39,955 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-04-09 07:53:39,986 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(139): Connect 0x3c1e7f16 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:39,991 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@672b468, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:40,008 DEBUG [hconnection-0x86f282a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:40,023 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42124, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:40,038 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:53:40,039 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(139): Connect 0x64b944ad to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:40,047 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33fc1e18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:40,047 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:53:40,076 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=169, AvailableMemoryMB=2402 2023-04-09 07:53:40,087 DEBUG [Listener at localhost/35977] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:53:40,094 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39416, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:53:40,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:53:40,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,110 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:53:40,112 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:53:40,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-04-09 07:53:40,116 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,117 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 empty. 2023-04-09 07:53:40,118 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,119 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-04-09 07:53:40,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-09 07:53:40,134 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:40,135 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5d73d913ad931f64b26893f58ee65a61, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:53:40,146 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:40,146 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 5d73d913ad931f64b26893f58ee65a61, disabling compactions & flushes 2023-04-09 07:53:40,146 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,146 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,147 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. after waiting 0 ms 2023-04-09 07:53:40,147 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,147 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,147 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:40,151 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:53:40,152 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026820152"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026820152"}]},"ts":"1681026820152"} 2023-04-09 07:53:40,155 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:53:40,157 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:53:40,157 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026820157"}]},"ts":"1681026820157"} 2023-04-09 07:53:40,159 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-04-09 07:53:40,162 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, ASSIGN}] 2023-04-09 07:53:40,164 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, ASSIGN 2023-04-09 07:53:40,166 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:53:40,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-09 07:53:40,317 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:40,318 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026820317"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026820317"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026820317"}]},"ts":"1681026820317"} 2023-04-09 07:53:40,321 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:40,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-09 07:53:40,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5d73d913ad931f64b26893f58ee65a61, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:40,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:40,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,483 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,485 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:40,485 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:40,488 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d73d913ad931f64b26893f58ee65a61 columnFamilyName family1 2023-04-09 07:53:40,491 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(310): Store=5d73d913ad931f64b26893f58ee65a61/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:40,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:40,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:53:40,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5d73d913ad931f64b26893f58ee65a61; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59707321, jitterRate=-0.11029158532619476}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:53:40,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:40,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., pid=11, masterSystemTime=1681026820473 2023-04-09 07:53:40,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,510 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:40,510 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026820510"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026820510"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026820510"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026820510"}]},"ts":"1681026820510"} 2023-04-09 07:53:40,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-04-09 07:53:40,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245 in 192 msec 2023-04-09 07:53:40,519 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-04-09 07:53:40,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, ASSIGN in 354 msec 2023-04-09 07:53:40,520 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:53:40,521 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026820521"}]},"ts":"1681026820521"} 2023-04-09 07:53:40,523 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-04-09 07:53:40,526 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:53:40,528 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 420 msec 2023-04-09 07:53:40,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-04-09 07:53:40,738 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-04-09 07:53:40,760 DEBUG [hconnection-0x611a5737-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:40,763 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42134, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:40,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:40,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,910 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:40,910 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:40,911 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:40,911 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:40,912 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,912 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,914 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:40,914 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:40,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:40,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:40,915 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:40,915 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:40,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,915 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:40,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,916 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,918 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:40,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:40,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:40,919 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:40,920 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:40,920 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:40,920 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:40,921 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:40,922 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:53:41,469 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a2f62d4ab9f94d33bca06b1325a389f7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2f62d4ab9f94d33bca06b1325a389f7 2023-04-09 07:53:41,890 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/17e2a91ed81146cfa616544d94a2eebe 2023-04-09 07:53:41,901 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/17e2a91ed81146cfa616544d94a2eebe as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe 2023-04-09 07:53:41,910 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe, entries=1000, sequenceid=5, filesize=175.8 K 2023-04-09 07:53:41,913 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 991ms, sequenceid=5, compaction requested=false 2023-04-09 07:53:41,914 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:41,917 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:41,917 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:41,917 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:41,918 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:41,920 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,920 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,921 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:41,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:41,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:41,923 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:41,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49385b5d[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:41,923 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,925 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:41,925 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:41,925 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:41,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:41,927 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:41,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:41,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:41,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:41,931 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:41,931 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:41,931 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:41,931 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,931 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:41,934 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,934 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,934 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:41,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:41,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:41,935 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,937 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:41,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:41,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:41,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:41,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:41,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:41,944 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:41,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:41,945 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:41,945 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:41,945 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:41,945 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:41,947 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:41,947 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:42,047 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:42,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:42,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:42,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:42,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,066 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:42,066 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:42,066 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:42,066 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:42,067 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,067 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,068 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:42,068 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:42,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:42,069 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,069 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:42,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,070 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:42,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,070 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:42,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,070 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:42,070 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:42,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:42,071 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:42,071 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:42,071 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:53:42,530 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283051ec64e0c8482f96a3ed66f5095107 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511283051ec64e0c8482f96a3ed66f5095107 2023-04-09 07:53:42,951 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/679a280f36f94628b37f5fcf9461792a 2023-04-09 07:53:42,969 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/679a280f36f94628b37f5fcf9461792a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a 2023-04-09 07:53:42,982 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:53:42,985 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 914ms, sequenceid=9, compaction requested=false 2023-04-09 07:53:42,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:42,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:42,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:42,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:42,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:42,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:42,989 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,989 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:42,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,990 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:42,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:42,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:42,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:42,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:42,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:42,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4873b653[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:42,994 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:42,994 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,001 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,002 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:43,002 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:43,002 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:43,002 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,002 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:43,007 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:43,008 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:43,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:43,014 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:43,014 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:43,014 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:43,014 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,014 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:43,020 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,020 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:43,020 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:43,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:43,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:43,023 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,027 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:43,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,035 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:43,035 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:43,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:43,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:43,036 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:43,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:43,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:43,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,037 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,037 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:43,037 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:43,037 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:43,137 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:43,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:43,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:43,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:43,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,162 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:43,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:43,163 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:43,163 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:43,164 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,164 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,166 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,166 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:43,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:43,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:43,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,167 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,167 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:43,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:43,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:43,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:43,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:43,171 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:43,172 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:43,172 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:43,172 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:43,172 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:43,172 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:43,407 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-04-09 07:53:43,409 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-04-09 07:53:43,662 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120122568c68c7f8414a9b59415e1ee2580b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120122568c68c7f8414a9b59415e1ee2580b 2023-04-09 07:53:43,708 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/c31d910d4f2f426cb7d82c2d657796b0 2023-04-09 07:53:43,722 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/c31d910d4f2f426cb7d82c2d657796b0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0 2023-04-09 07:53:43,737 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0, entries=1, sequenceid=13, filesize=5.0 K 2023-04-09 07:53:43,738 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 566ms, sequenceid=13, compaction requested=true 2023-04-09 07:53:43,738 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:43,738 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:43,739 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:43,742 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,742 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:43,743 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,743 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:43,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:43,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:43,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ee01b17[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:43,746 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:43,746 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,747 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:43,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:43,748 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,748 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:43,751 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:43,751 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,751 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:43,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:43,751 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:43,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:43,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:43,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:43,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:43,758 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:43,758 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:43,758 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:43,758 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,758 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:43,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:43,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:43,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:43,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:43,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:43,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:43,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,762 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,764 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:43,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:43,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:43,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,778 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:43,779 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,779 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,779 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,779 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:43,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:43,779 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:43,880 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:43,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:43,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:43,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:43,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,896 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:43,896 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:43,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:43,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:43,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,899 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:43,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:43,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:43,899 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,900 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:43,900 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:43,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:43,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,901 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,902 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:43,902 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:43,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:43,903 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:43,903 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:43,903 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:43,903 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:43,960 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056f579fbad3b444bbb6b53617c25c140f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056f579fbad3b444bbb6b53617c25c140f 2023-04-09 07:53:44,185 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:53:44,383 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/f7767e9900da4038ba7b5a53182a09f6 2023-04-09 07:53:44,395 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/f7767e9900da4038ba7b5a53182a09f6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6 2023-04-09 07:53:44,408 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6, entries=1, sequenceid=17, filesize=5.0 K 2023-04-09 07:53:44,411 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 508ms, sequenceid=17, compaction requested=true 2023-04-09 07:53:44,411 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:44,411 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:44,411 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:44,413 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:44,417 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:44,420 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:44,421 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,421 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:44,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:44,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:44,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:44,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4541c7e1[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:44,424 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:44,424 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,426 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,426 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:44,426 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:44,427 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:44,429 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,429 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:44,430 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:44,431 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:44,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:44,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:44,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:44,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:44,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:44,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:44,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:44,436 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:44,436 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:44,436 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:44,436 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,437 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:44,439 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,439 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:44,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,439 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:44,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:44,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:44,440 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:44,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,444 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:44,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:44,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:44,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,452 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:44,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:44,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:44,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:44,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:44,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:44,454 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:44,454 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:44,554 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:44,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:44,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:44,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:44,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,579 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:44,579 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:44,580 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:44,580 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:44,580 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,580 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,582 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:44,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:44,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,585 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:44,585 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,585 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,586 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:44,586 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:44,586 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:44,586 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:44,586 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:44,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:44,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:44,590 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:44,590 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:44,590 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:45,042 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228700e0da75d7846f3b5eb48e636d8d31d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228700e0da75d7846f3b5eb48e636d8d31d 2023-04-09 07:53:45,072 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/a591069b81434aa4933182b30a49baf9 2023-04-09 07:53:45,089 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/a591069b81434aa4933182b30a49baf9 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9 2023-04-09 07:53:45,100 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9, entries=1, sequenceid=21, filesize=5.0 K 2023-04-09 07:53:45,102 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 512ms, sequenceid=21, compaction requested=true 2023-04-09 07:53:45,102 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:45,103 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:45,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:45,113 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,114 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,114 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:45,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:45,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ea88ea7[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:45,117 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:45,117 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,120 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,120 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:45,120 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:45,120 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:45,120 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,120 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:45,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:45,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:45,129 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:45,129 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:45,129 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:45,129 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,129 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:45,132 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,132 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:45,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,132 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:45,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,160 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:45,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:45,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,161 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:45,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:45,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:45,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:45,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:45,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,163 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:45,163 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:45,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:45,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:45,164 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-09 07:53:45,164 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-04-09 07:53:45,263 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:45,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:45,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:45,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:45,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,291 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:45,291 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:45,291 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:45,291 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:45,292 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,292 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,294 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,294 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:45,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:45,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:45,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:45,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:45,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,296 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:45,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:45,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:45,312 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:45,316 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:45,316 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:45,316 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:45,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123198a71ebdd8774ab3abc50616bbfdbc5a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123198a71ebdd8774ab3abc50616bbfdbc5a 2023-04-09 07:53:45,806 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/e41148eb15124fe1b47543031c01c87e 2023-04-09 07:53:45,823 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/e41148eb15124fe1b47543031c01c87e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e 2023-04-09 07:53:45,835 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e, entries=1, sequenceid=25, filesize=5.0 K 2023-04-09 07:53:45,836 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 520ms, sequenceid=25, compaction requested=true 2023-04-09 07:53:45,837 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:45,837 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:45,837 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:45,838 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:45,842 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b3c780f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:45,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:45,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,846 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,846 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:45,846 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:45,846 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:45,847 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,847 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:45,848 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:45,848 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:45,848 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:45,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:45,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:45,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:45,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:45,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:45,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:45,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:45,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:45,856 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,858 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:45,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:45,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:45,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:45,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:45,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:45,862 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:45,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:45,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:45,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:45,864 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:45,864 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:45,964 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:45,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:45,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:45,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:45,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,981 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:45,981 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:45,982 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:45,982 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:45,983 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,983 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,984 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:45,984 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:45,985 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:45,985 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:45,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:45,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:45,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:45,987 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:45,988 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:45,994 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:45,994 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:45,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:45,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:45,995 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:46,096 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010129044f46cba04c4fb8aa920ca5421e3a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129044f46cba04c4fb8aa920ca5421e3a 2023-04-09 07:53:46,111 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/bab308d4fac84a38a9bacf6b32c0def5 2023-04-09 07:53:46,120 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/bab308d4fac84a38a9bacf6b32c0def5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5 2023-04-09 07:53:46,133 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5, entries=1, sequenceid=29, filesize=5.0 K 2023-04-09 07:53:46,135 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 140ms, sequenceid=29, compaction requested=true 2023-04-09 07:53:46,135 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:46,135 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:46,138 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:46,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:46,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75eaf0dc[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:46,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:46,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,144 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:46,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:46,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:46,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:46,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:46,146 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:46,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:46,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:46,150 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:46,151 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:46,151 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:46,151 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,151 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:46,153 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,153 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,153 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:46,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:46,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,155 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,157 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,163 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:46,163 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:46,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:46,163 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:46,163 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,163 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:46,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:46,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:46,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,165 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:46,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:46,165 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:46,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:46,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:46,265 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:46,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:46,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:46,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:46,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,282 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:46,282 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:46,287 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:46,287 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:46,288 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,288 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,290 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:46,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:46,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:46,290 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,290 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:46,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:46,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:46,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:46,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:46,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:46,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:46,295 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:46,295 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:46,295 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:46,295 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:46,335 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032c228aefc07941fa83ec29651ba068c7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601032c228aefc07941fa83ec29651ba068c7 2023-04-09 07:53:46,799 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/d94ed1cf9e0a4ed5996954b1acb1728c 2023-04-09 07:53:46,813 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/d94ed1cf9e0a4ed5996954b1acb1728c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c 2023-04-09 07:53:46,823 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c, entries=1, sequenceid=33, filesize=5.0 K 2023-04-09 07:53:46,824 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 529ms, sequenceid=33, compaction requested=true 2023-04-09 07:53:46,825 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:46,825 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:46,825 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:46,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:46,828 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:46,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e5f4f4f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:46,842 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:46,842 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,844 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:46,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:46,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:46,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:46,847 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:46,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:46,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:46,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:46,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:46,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:46,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:46,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:46,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:46,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:46,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:46,855 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:46,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:46,858 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:46,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:46,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:46,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:46,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:46,862 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:46,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:46,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,863 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:46,864 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:46,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:46,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:46,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:46,964 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:46,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:46,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:46,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:46,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,979 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:46,979 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:46,980 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:46,980 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:46,980 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,980 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,983 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:46,983 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:46,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:46,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,984 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:46,984 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:46,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:46,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,985 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:46,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:46,988 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:46,988 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:46,988 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:46,988 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:46,989 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:46,989 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:46,989 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:46,989 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:47,447 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907929e63f465984c3ea8448759564b43c0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907929e63f465984c3ea8448759564b43c0 2023-04-09 07:53:47,457 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/475d06ea4b914b1b88bd0888e1ba1a6d 2023-04-09 07:53:47,469 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/475d06ea4b914b1b88bd0888e1ba1a6d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d 2023-04-09 07:53:47,478 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d, entries=1, sequenceid=37, filesize=5.0 K 2023-04-09 07:53:47,479 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 490ms, sequenceid=37, compaction requested=true 2023-04-09 07:53:47,480 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:47,480 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:47,480 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:47,484 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,485 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,489 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,489 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:47,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:47,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34083395[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:47,494 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:47,494 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,495 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,496 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:47,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:47,496 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:47,496 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:47,498 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,498 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:47,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:47,508 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:47,508 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:47,508 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:47,508 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,508 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:47,510 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,510 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:47,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,510 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:47,511 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:47,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,518 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:47,518 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:47,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:47,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:47,518 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,518 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,519 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,519 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,519 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,519 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,519 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,519 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:47,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:47,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:47,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:47,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:47,520 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:47,520 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:47,530 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:47,620 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:47,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:47,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:47,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:47,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,642 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:47,642 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:47,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-04-09 07:53:47,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-04-09 07:53:47,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,644 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,645 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:47,646 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:47,646 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,646 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:47,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:47,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-04-09 07:53:47,648 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,648 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-04-09 07:53:47,648 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-04-09 07:53:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:47,649 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:47,649 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:47,649 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. started... 2023-04-09 07:53:47,650 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5d73d913ad931f64b26893f58ee65a61 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:47,684 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112073e23d27345940f4bdd5a78b1b52169f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112073e23d27345940f4bdd5a78b1b52169f 2023-04-09 07:53:47,701 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/ca9da995cd544e1bbb44145c1b1c2903 2023-04-09 07:53:47,711 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/ca9da995cd544e1bbb44145c1b1c2903 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903 2023-04-09 07:53:47,722 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903, entries=1, sequenceid=41, filesize=5.0 K 2023-04-09 07:53:47,726 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5d73d913ad931f64b26893f58ee65a61 in 77ms, sequenceid=41, compaction requested=true 2023-04-09 07:53:47,727 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:47,727 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:47,727 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:47,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:47,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-04-09 07:53:47,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-04-09 07:53:47,730 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,730 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,730 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,731 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:47,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-04-09 07:53:47,732 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c8cc91b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:47,732 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-04-09 07:53:47,732 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,734 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,734 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-04-09 07:53:47,734 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-04-09 07:53:47,734 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:47,735 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,735 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:47,745 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-04-09 07:53:47,745 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,745 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:47,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,746 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-04-09 07:53:47,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:47,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-04-09 07:53:47,752 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-04-09 07:53:47,752 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:47,752 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:47,752 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,752 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:47,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:47,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:47,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:47,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:47,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,763 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,765 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:47,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,777 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:47,778 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:47,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:47,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:47,778 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:47,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-04-09 07:53:47,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:47,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:47,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:47,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-04-09 07:53:47,779 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:47,779 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:47,879 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:47,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:47,887 DEBUG [Listener at localhost/35977] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-09 07:53:47,890 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42940, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-09 07:53:47,892 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:47,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:47,923 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:47,925 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027427925 2023-04-09 07:53:47,925 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:47,927 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:53:47,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-04-09 07:53:47,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:47,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x77d62183 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:47,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c864339, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:47,948 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,505 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010339a63f95fedf4e5fbd233b8efd3fce72 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010339a63f95fedf4e5fbd233b8efd3fce72 2023-04-09 07:53:48,507 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:48,531 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:48,534 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39856, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:48,554 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:53:48,556 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:53:48,619 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/35f7fa95dbbf4ad49af2a705e2d9f744 first=Optional[r3] last=Optional[r6] 2023-04-09 07:53:48,636 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/35f7fa95dbbf4ad49af2a705e2d9f744}] 2023-04-09 07:53:48,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/35f7fa95dbbf4ad49af2a705e2d9f744 for inclusion in 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:48,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-09 07:53:48,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:48,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:48,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/35f7fa95dbbf4ad49af2a705e2d9f744 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__so07443c7t17f5iuq6u5dbcqk9utlsk44kjjfck0dqqpds40405p756ke20iuulv/family1/35f7fa95dbbf4ad49af2a705e2d9f744 2023-04-09 07:53:48,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__so07443c7t17f5iuq6u5dbcqk9utlsk44kjjfck0dqqpds40405p756ke20iuulv/family1/35f7fa95dbbf4ad49af2a705e2d9f744 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_ 2023-04-09 07:53:48,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__so07443c7t17f5iuq6u5dbcqk9utlsk44kjjfck0dqqpds40405p756ke20iuulv/family1/35f7fa95dbbf4ad49af2a705e2d9f744 into 5d73d913ad931f64b26893f58ee65a61/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_ - updating store file list. 2023-04-09 07:53:48,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_ into 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:48,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__so07443c7t17f5iuq6u5dbcqk9utlsk44kjjfck0dqqpds40405p756ke20iuulv/family1/35f7fa95dbbf4ad49af2a705e2d9f744 into 5d73d913ad931f64b26893f58ee65a61/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_) 2023-04-09 07:53:48,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__so07443c7t17f5iuq6u5dbcqk9utlsk44kjjfck0dqqpds40405p756ke20iuulv/family1/35f7fa95dbbf4ad49af2a705e2d9f744 2023-04-09 07:53:48,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:53:48,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:53:48,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 5d73d913ad931f64b26893f58ee65a61/family1 is initiating minor compaction 2023-04-09 07:53:48,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@70b35447; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:48,706 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5d73d913ad931f64b26893f58ee65a61/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:48,707 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp, totalSize=50.2 K 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 679a280f36f94628b37f5fcf9461792a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting c31d910d4f2f426cb7d82c2d657796b0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting f7767e9900da4038ba7b5a53182a09f6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting a591069b81434aa4933182b30a49baf9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting e41148eb15124fe1b47543031c01c87e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting bab308d4fac84a38a9bacf6b32c0def5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting d94ed1cf9e0a4ed5996954b1acb1728c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 475d06ea4b914b1b88bd0888e1ba1a6d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting ca9da995cd544e1bbb44145c1b1c2903, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:53:48,708 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:53:48,725 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:48,728 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:48,737 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228700e0da75d7846f3b5eb48e636d8d31d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228700e0da75d7846f3b5eb48e636d8d31d 2023-04-09 07:53:48,743 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123198a71ebdd8774ab3abc50616bbfdbc5a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015123198a71ebdd8774ab3abc50616bbfdbc5a 2023-04-09 07:53:48,749 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129044f46cba04c4fb8aa920ca5421e3a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129044f46cba04c4fb8aa920ca5421e3a 2023-04-09 07:53:48,752 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601032c228aefc07941fa83ec29651ba068c7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601032c228aefc07941fa83ec29651ba068c7 2023-04-09 07:53:48,752 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-04-09 07:53:48,752 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,753 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:48,806 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5d73d913ad931f64b26893f58ee65a61#family1#compaction#10 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:53:48,949 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205260dbbc8ec824649a600ba92701e373a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205260dbbc8ec824649a600ba92701e373a 2023-04-09 07:53:48,951 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:49,019 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2a34d87d54745869c5dcf6bb4158e6f first=Optional[r00] last=Optional[r2] 2023-04-09 07:53:49,022 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2a34d87d54745869c5dcf6bb4158e6f}] 2023-04-09 07:53:49,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2a34d87d54745869c5dcf6bb4158e6f for inclusion in 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:49,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-09 07:53:49,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:49,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:49,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b2a34d87d54745869c5dcf6bb4158e6f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__79bqlfrdk7t5hbodalhf1e3ivck5l9jrb2cla08c1e3f70f7dmelle4s12413ipt/family1/b2a34d87d54745869c5dcf6bb4158e6f 2023-04-09 07:53:49,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__79bqlfrdk7t5hbodalhf1e3ivck5l9jrb2cla08c1e3f70f7dmelle4s12413ipt/family1/b2a34d87d54745869c5dcf6bb4158e6f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ 2023-04-09 07:53:49,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__79bqlfrdk7t5hbodalhf1e3ivck5l9jrb2cla08c1e3f70f7dmelle4s12413ipt/family1/b2a34d87d54745869c5dcf6bb4158e6f into 5d73d913ad931f64b26893f58ee65a61/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ - updating store file list. 2023-04-09 07:53:49,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ into 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:49,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__79bqlfrdk7t5hbodalhf1e3ivck5l9jrb2cla08c1e3f70f7dmelle4s12413ipt/family1/b2a34d87d54745869c5dcf6bb4158e6f into 5d73d913ad931f64b26893f58ee65a61/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_) 2023-04-09 07:53:49,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__79bqlfrdk7t5hbodalhf1e3ivck5l9jrb2cla08c1e3f70f7dmelle4s12413ipt/family1/b2a34d87d54745869c5dcf6bb4158e6f 2023-04-09 07:53:49,056 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:49,057 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:49,061 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2f62d4ab9f94d33bca06b1325a389f7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a2f62d4ab9f94d33bca06b1325a389f7 2023-04-09 07:53:49,065 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120122568c68c7f8414a9b59415e1ee2580b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120122568c68c7f8414a9b59415e1ee2580b 2023-04-09 07:53:49,067 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056f579fbad3b444bbb6b53617c25c140f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512056f579fbad3b444bbb6b53617c25c140f 2023-04-09 07:53:49,067 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-04-09 07:53:49,067 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-09 07:53:49,068 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:49,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:49,069 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:49,070 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:49,072 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.1560 sec 2023-04-09 07:53:49,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-09 07:53:49,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:49,398 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-04-09 07:53:49,403 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:49,403 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-04-09 07:53:49,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-04-09 07:53:49,434 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, REOPEN/MOVE}] 2023-04-09 07:53:49,435 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, REOPEN/MOVE 2023-04-09 07:53:49,437 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:49,437 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026829437"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026829437"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026829437"}]},"ts":"1681026829437"} 2023-04-09 07:53:49,440 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:49,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:49,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5d73d913ad931f64b26893f58ee65a61, disabling compactions & flushes 2023-04-09 07:53:49,616 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:49,701 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/fe5372857306425ea86717b22cd43eaa as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa 2023-04-09 07:53:49,717 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5d73d913ad931f64b26893f58ee65a61/family1 of 5d73d913ad931f64b26893f58ee65a61 into fe5372857306425ea86717b22cd43eaa(size=6.6 K), total size for store is 358.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:53:49,717 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:49,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:49,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:49,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. after waiting 0 ms 2023-04-09 07:53:49,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:49,719 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1681026828703; duration=1sec 2023-04-09 07:53:49,719 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:49,730 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_] to archive 2023-04-09 07:53:49,731 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:49,736 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/679a280f36f94628b37f5fcf9461792a 2023-04-09 07:53:49,744 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/c31d910d4f2f426cb7d82c2d657796b0 2023-04-09 07:53:49,748 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/f7767e9900da4038ba7b5a53182a09f6 2023-04-09 07:53:49,752 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a591069b81434aa4933182b30a49baf9 2023-04-09 07:53:49,754 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/e41148eb15124fe1b47543031c01c87e 2023-04-09 07:53:49,757 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/bab308d4fac84a38a9bacf6b32c0def5 2023-04-09 07:53:49,758 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/d94ed1cf9e0a4ed5996954b1acb1728c 2023-04-09 07:53:49,760 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/475d06ea4b914b1b88bd0888e1ba1a6d 2023-04-09 07:53:49,765 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/ca9da995cd544e1bbb44145c1b1c2903 2023-04-09 07:53:49,769 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/61a2a2bd6bd54594b513074e0b3dd1cb_SeqId_44_ 2023-04-09 07:53:49,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-04-09 07:53:49,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:49,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:49,819 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 5d73d913ad931f64b26893f58ee65a61 to self. 2023-04-09 07:53:49,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:49,823 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=CLOSED 2023-04-09 07:53:49,824 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026829823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026829823"}]},"ts":"1681026829823"} 2023-04-09 07:53:49,829 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-04-09 07:53:49,829 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245 in 386 msec 2023-04-09 07:53:49,830 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=true 2023-04-09 07:53:49,981 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:49,982 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026829981"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026829981"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026829981"}]},"ts":"1681026829981"} 2023-04-09 07:53:49,985 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:50,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:50,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5d73d913ad931f64b26893f58ee65a61, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:50,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:50,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,145 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,146 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:50,146 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:50,147 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5d73d913ad931f64b26893f58ee65a61 columnFamilyName family1 2023-04-09 07:53:50,157 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe 2023-04-09 07:53:50,169 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ 2023-04-09 07:53:50,179 DEBUG [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa 2023-04-09 07:53:50,180 INFO [StoreOpener-5d73d913ad931f64b26893f58ee65a61-1] regionserver.HStore(310): Store=5d73d913ad931f64b26893f58ee65a61/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:50,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:53:50,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5d73d913ad931f64b26893f58ee65a61; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67377372, jitterRate=0.0040010809898376465}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:53:50,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:50,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., pid=17, masterSystemTime=1681026830137 2023-04-09 07:53:50,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:50,214 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:53:50,216 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:53:50,216 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1912): 5d73d913ad931f64b26893f58ee65a61/family1 is initiating minor compaction (all files) 2023-04-09 07:53:50,217 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5d73d913ad931f64b26893f58ee65a61/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:50,217 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp, totalSize=358.6 K 2023-04-09 07:53:50,218 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 17e2a91ed81146cfa616544d94a2eebe, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:53:50,218 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting fe5372857306425ea86717b22cd43eaa, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:53:50,219 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5d73d913ad931f64b26893f58ee65a61, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:50,219 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 6d7677b4879e47759c0700a6a00d1989_SeqId_46_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878840049 2023-04-09 07:53:50,219 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1681026830219"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026830219"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026830219"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026830219"}]},"ts":"1681026830219"} 2023-04-09 07:53:50,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:50,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:53:50,226 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-04-09 07:53:50,227 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 5d73d913ad931f64b26893f58ee65a61, server=jenkins-hbase4.apache.org,34907,1681026816245 in 238 msec 2023-04-09 07:53:50,229 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-04-09 07:53:50,230 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=5d73d913ad931f64b26893f58ee65a61, REOPEN/MOVE in 793 msec 2023-04-09 07:53:50,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-04-09 07:53:50,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 818 msec 2023-04-09 07:53:50,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 891 msec 2023-04-09 07:53:50,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-04-09 07:53:50,274 INFO [Listener at localhost/35977] compactions.TestMobCompactor(278): alter status finished 2023-04-09 07:53:50,281 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-04-09 07:53:50,283 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:50,288 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:50,292 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5d73d913ad931f64b26893f58ee65a61#family1#compaction#11 average throughput is 2.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:53:50,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:50,294 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027430294 2023-04-09 07:53:50,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:53:50,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-04-09 07:53:50,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:50,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x34313c3e to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:50,342 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@25fc4714, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:50,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,426 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f38dcabbfc8a433bb5160aef61868e4d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128f38dcabbfc8a433bb5160aef61868e4d 2023-04-09 07:53:50,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:50,463 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:50,464 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39870, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:50,479 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:53:50,482 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:53:50,521 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5348c71978b94c8bb6dc410ea75c25a0 first=Optional[r05] last=Optional[r8] 2023-04-09 07:53:50,522 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=50 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5348c71978b94c8bb6dc410ea75c25a0}] 2023-04-09 07:53:50,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5348c71978b94c8bb6dc410ea75c25a0 for inclusion in 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:50,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-09 07:53:50,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:50,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:50,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/5348c71978b94c8bb6dc410ea75c25a0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gd3jlll27fmdhidaqrglo2vel679k1ph6diqgpjgg32lq2d6d3nhhajklffg9bd5/family1/5348c71978b94c8bb6dc410ea75c25a0 2023-04-09 07:53:50,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gd3jlll27fmdhidaqrglo2vel679k1ph6diqgpjgg32lq2d6d3nhhajklffg9bd5/family1/5348c71978b94c8bb6dc410ea75c25a0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/68971f630fcf4056ba0ffedb8116d660_SeqId_52_ 2023-04-09 07:53:50,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gd3jlll27fmdhidaqrglo2vel679k1ph6diqgpjgg32lq2d6d3nhhajklffg9bd5/family1/5348c71978b94c8bb6dc410ea75c25a0 into 5d73d913ad931f64b26893f58ee65a61/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/68971f630fcf4056ba0ffedb8116d660_SeqId_52_ - updating store file list. 2023-04-09 07:53:50,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/68971f630fcf4056ba0ffedb8116d660_SeqId_52_ into 5d73d913ad931f64b26893f58ee65a61/family1 2023-04-09 07:53:50,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gd3jlll27fmdhidaqrglo2vel679k1ph6diqgpjgg32lq2d6d3nhhajklffg9bd5/family1/5348c71978b94c8bb6dc410ea75c25a0 into 5d73d913ad931f64b26893f58ee65a61/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/68971f630fcf4056ba0ffedb8116d660_SeqId_52_) 2023-04-09 07:53:50,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__gd3jlll27fmdhidaqrglo2vel679k1ph6diqgpjgg32lq2d6d3nhhajklffg9bd5/family1/5348c71978b94c8bb6dc410ea75c25a0 2023-04-09 07:53:50,568 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:50,570 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:50,577 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112073e23d27345940f4bdd5a78b1b52169f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112073e23d27345940f4bdd5a78b1b52169f 2023-04-09 07:53:50,580 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511283051ec64e0c8482f96a3ed66f5095107 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511283051ec64e0c8482f96a3ed66f5095107 2023-04-09 07:53:50,580 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-04-09 07:53:50,580 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-09 07:53:50,581 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,581 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,581 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-04-09 07:53:50,581 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-04-09 07:53:50,581 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:50,582 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-04-09 07:53:50,587 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 301 msec 2023-04-09 07:53:50,746 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=255 (was 210) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1031621008_15 at /127.0.0.1:38098 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:34907-shortCompactions-0 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:940) org.apache.hadoop.hdfs.DFSOutputStream.completeFile(DFSOutputStream.java:880) org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:863) org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:818) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72) org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishClose(HFileWriterImpl.java:820) org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.close(HFileWriterImpl.java:654) org.apache.hadoop.hbase.regionserver.StoreFileWriter.close(StoreFileWriter.java:359) org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.abortWriter(DefaultCompactor.java:97) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(DefaultMobStoreCompactor.java:339) org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:337) org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:108) org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:122) org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1473) org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2267) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625) org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120592ea84b6497b4603adcf0584bd3820a1_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38062 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38040 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x7e790efc-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x77d3998b-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38188 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38210 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38230 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x77d62183-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1755355660-172.31.14.131-1681026811671:blk_1073741865_1041, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38262 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-1755355660-172.31.14.131-1681026811671:blk_1073741865_1041 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1031621008_15 at /127.0.0.1:38280 [Receiving block BP-1755355660-172.31.14.131-1681026811671:blk_1073741865_1041] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x77d3998b-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e790efc-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x77d3998b-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Thread-300 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38090 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x77d3998b-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x77d62183-SendThread(127.0.0.1:57867) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x34313c3e-SendThread(127.0.0.1:57867) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38154 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/a7726ecf1592443d9127de6edc035a69 block BP-1755355660-172.31.14.131-1681026811671:blk_1073741865_1041 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38202 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38198 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38010 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e790efc-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x34313c3e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x77d62183 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/347772476.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x77d3998b-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x34313c3e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/347772476.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38118 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38074 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38274 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=601 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=331 (was 252) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1997 (was 2402) 2023-04-09 07:53:50,764 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=255, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=331, ProcessCount=169, AvailableMemoryMB=1994 2023-04-09 07:53:50,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-09 07:53:50,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:50,770 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:53:50,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-04-09 07:53:50,771 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:53:50,773 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:50,774 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6 empty. 2023-04-09 07:53:50,774 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:50,774 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-04-09 07:53:50,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:50,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:51,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:51,173 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/.tmp/family1/a7726ecf1592443d9127de6edc035a69 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/a7726ecf1592443d9127de6edc035a69 2023-04-09 07:53:51,185 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5d73d913ad931f64b26893f58ee65a61/family1 of 5d73d913ad931f64b26893f58ee65a61 into a7726ecf1592443d9127de6edc035a69(size=177.2 K), total size for store is 182.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:53:51,186 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:53:51,186 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61., storeName=5d73d913ad931f64b26893f58ee65a61/family1, priority=13, startTime=1681026830213; duration=0sec 2023-04-09 07:53:51,186 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:51,197 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:51,212 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => fced19ac59ac2562418987229fbaf2d6, NAME => 'testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:53:51,248 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_] to archive 2023-04-09 07:53:51,250 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:51,262 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/17e2a91ed81146cfa616544d94a2eebe 2023-04-09 07:53:51,264 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/fe5372857306425ea86717b22cd43eaa 2023-04-09 07:53:51,266 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/family1/6d7677b4879e47759c0700a6a00d1989_SeqId_46_ 2023-04-09 07:53:51,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:51,645 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:51,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing fced19ac59ac2562418987229fbaf2d6, disabling compactions & flushes 2023-04-09 07:53:51,646 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:51,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:51,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. after waiting 0 ms 2023-04-09 07:53:51,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:51,646 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:51,646 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:51,656 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:53:51,658 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026831657"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026831657"}]},"ts":"1681026831657"} 2023-04-09 07:53:51,662 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:53:51,669 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:53:51,669 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026831669"}]},"ts":"1681026831669"} 2023-04-09 07:53:51,672 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-04-09 07:53:51,678 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fced19ac59ac2562418987229fbaf2d6, ASSIGN}] 2023-04-09 07:53:51,680 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fced19ac59ac2562418987229fbaf2d6, ASSIGN 2023-04-09 07:53:51,682 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fced19ac59ac2562418987229fbaf2d6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:53:51,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fced19ac59ac2562418987229fbaf2d6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:51,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026831833"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026831833"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026831833"}]},"ts":"1681026831833"} 2023-04-09 07:53:51,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure fced19ac59ac2562418987229fbaf2d6, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:53:51,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:51,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fced19ac59ac2562418987229fbaf2d6, NAME => 'testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:53:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:53:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:51,996 INFO [StoreOpener-fced19ac59ac2562418987229fbaf2d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:51,998 DEBUG [StoreOpener-fced19ac59ac2562418987229fbaf2d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:51,998 DEBUG [StoreOpener-fced19ac59ac2562418987229fbaf2d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:51,999 INFO [StoreOpener-fced19ac59ac2562418987229fbaf2d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fced19ac59ac2562418987229fbaf2d6 columnFamilyName family1 2023-04-09 07:53:52,001 INFO [StoreOpener-fced19ac59ac2562418987229fbaf2d6-1] regionserver.HStore(310): Store=fced19ac59ac2562418987229fbaf2d6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:53:52,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:52,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:52,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:53:52,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:53:52,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fced19ac59ac2562418987229fbaf2d6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67751565, jitterRate=0.009576991200447083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:53:52,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:52,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., pid=21, masterSystemTime=1681026831989 2023-04-09 07:53:52,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:52,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:52,023 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fced19ac59ac2562418987229fbaf2d6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:52,023 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026832023"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026832023"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026832023"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026832023"}]},"ts":"1681026832023"} 2023-04-09 07:53:52,032 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-04-09 07:53:52,032 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure fced19ac59ac2562418987229fbaf2d6, server=jenkins-hbase4.apache.org,34907,1681026816245 in 192 msec 2023-04-09 07:53:52,035 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-04-09 07:53:52,035 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=fced19ac59ac2562418987229fbaf2d6, ASSIGN in 354 msec 2023-04-09 07:53:52,036 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:53:52,037 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026832036"}]},"ts":"1681026832036"} 2023-04-09 07:53:52,039 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-04-09 07:53:52,042 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:53:52,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.2750 sec 2023-04-09 07:53:52,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-09 07:53:52,890 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-04-09 07:53:52,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:52,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,944 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:52,944 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:52,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:52,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:52,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,947 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:52,947 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:52,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:52,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:52,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,948 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:52,948 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:52,949 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:52,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,960 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:52,960 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:52,961 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:52,961 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:52,961 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:52,961 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:52,961 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:52,962 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:53:53,019 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113009fb2ef4499b4d0b888c2ef8da9ad07f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113009fb2ef4499b4d0b888c2ef8da9ad07f 2023-04-09 07:53:53,424 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/db18950f4ef9465f93763da6fbef6f39 2023-04-09 07:53:53,439 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/db18950f4ef9465f93763da6fbef6f39 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39 2023-04-09 07:53:53,448 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39, entries=1000, sequenceid=5, filesize=166.0 K 2023-04-09 07:53:53,449 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 487ms, sequenceid=5, compaction requested=false 2023-04-09 07:53:53,449 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:53,450 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:53,450 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:53,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:53,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:53,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:53,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:53,452 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,452 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:53,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:53,463 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,463 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:53,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:53,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:53,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:53,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:53,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ec2bc46[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:53,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:53,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,467 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:53,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:53,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:53,468 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,468 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:53,470 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:53,470 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:53,470 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,470 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:53,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:53,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:53,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:53,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:53,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:53,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:53,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:53,474 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:53,474 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:53,474 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:53,474 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,474 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:53,475 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,476 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:53,476 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:53,476 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:53,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:53,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,478 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:53,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,481 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:53,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:53,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:53,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:53,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:53,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:53,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:53,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:53,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:53,482 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:53,483 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:53,583 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:53,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:53,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:53,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:53,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,593 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:53,594 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:53,594 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:53,594 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:53,595 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,595 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,596 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:53,596 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:53,597 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:53,597 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,598 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,598 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:53,598 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:53,599 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:53,599 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:53,599 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:53,599 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:53,599 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:53,599 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:53:54,020 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128da57c6949d524625b184f12f19a2a549 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128da57c6949d524625b184f12f19a2a549 2023-04-09 07:53:54,437 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/df2412df21584b3eb1962b4c23740cb0 2023-04-09 07:53:54,446 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/df2412df21584b3eb1962b4c23740cb0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0 2023-04-09 07:53:54,452 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:53:54,453 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 854ms, sequenceid=9, compaction requested=false 2023-04-09 07:53:54,454 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:54,454 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:54,454 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:54,454 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:54,454 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,454 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:54,454 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:54,456 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,456 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,457 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,457 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:54,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aa3a3e6[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:54,459 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:54,459 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,460 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,460 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:54,460 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:54,460 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:54,460 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,460 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:54,465 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:54,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:54,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:54,469 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:54,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:54,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:54,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,469 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:54,471 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,471 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,471 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,472 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,473 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,479 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,479 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:54,479 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:54,479 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:54,479 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:54,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:54,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:54,480 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,480 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,480 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,481 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,481 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:54,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,481 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:54,581 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:54,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:54,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:54,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:54,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,592 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:54,592 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:54,592 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:54,592 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:54,599 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,599 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,600 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:54,600 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:54,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:54,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,601 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,601 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:54,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:54,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,602 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,603 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:54,603 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,603 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:54,603 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:54,604 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:54,604 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:54,604 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:54,604 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:54,623 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201af30bcf0258f4827a0a9c65ac39e9a1e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201af30bcf0258f4827a0a9c65ac39e9a1e 2023-04-09 07:53:54,633 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/91f32b16a0394f6fb2e7b1ae474a7734 2023-04-09 07:53:54,643 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/91f32b16a0394f6fb2e7b1ae474a7734 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734 2023-04-09 07:53:54,651 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734, entries=1, sequenceid=13, filesize=4.9 K 2023-04-09 07:53:54,652 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 48ms, sequenceid=13, compaction requested=true 2023-04-09 07:53:54,652 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:54,652 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:54,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:54,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:54,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:54,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:54,654 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,654 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:54,657 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f42a9bc[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:54,657 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:54,658 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,659 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:54,659 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:54,659 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,659 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:54,665 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:54,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,665 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:54,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:54,674 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:54,674 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:54,674 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:54,674 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,674 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:54,676 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,676 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,676 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,678 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,679 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:54,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:54,682 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:54,683 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,683 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,684 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:54,684 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:54,784 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:54,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:54,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:54,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:54,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,794 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:54,794 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:54,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:54,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:54,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,797 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:54,797 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:54,797 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,797 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:54,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,798 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:54,799 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:54,800 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:54,813 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:54,813 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:54,813 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:54,813 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:54,850 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120594082eccca4445ecbd2f387190ab984b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120594082eccca4445ecbd2f387190ab984b 2023-04-09 07:53:54,878 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/bd0dd891df7f4c58a4cd21d015e73c74 2023-04-09 07:53:54,884 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/bd0dd891df7f4c58a4cd21d015e73c74 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74 2023-04-09 07:53:54,893 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74, entries=1, sequenceid=17, filesize=4.9 K 2023-04-09 07:53:54,894 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 81ms, sequenceid=17, compaction requested=true 2023-04-09 07:53:54,894 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:54,895 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:54,897 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,897 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:54,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:54,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75ac775e[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:54,900 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:54,900 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,901 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,902 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,902 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:54,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:54,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:54,902 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:54,903 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:54,903 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,903 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,904 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:54,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:54,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:54,908 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:54,908 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:54,908 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:54,908 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,908 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:54,910 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,910 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:54,910 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:54,911 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:54,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,920 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:54,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:54,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:54,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:54,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:54,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:54,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:54,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:54,927 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:54,927 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:55,027 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:55,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:55,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:55,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:55,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,037 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:55,037 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:55,038 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:55,038 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:55,039 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,039 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,041 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:55,041 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:55,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:55,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:55,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:55,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:55,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,054 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:55,054 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:55,055 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:55,055 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:55,055 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:55,055 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:55,056 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:55,056 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:55,148 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c5cd511daad142498ee079d9328fa61c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5cd511daad142498ee079d9328fa61c 2023-04-09 07:53:55,362 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:53:55,566 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/c2bef33e103445568766c8d9f710528c 2023-04-09 07:53:55,573 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/c2bef33e103445568766c8d9f710528c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c 2023-04-09 07:53:55,579 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c, entries=1, sequenceid=21, filesize=4.9 K 2023-04-09 07:53:55,580 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 524ms, sequenceid=21, compaction requested=true 2023-04-09 07:53:55,581 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:55,581 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:55,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:55,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:55,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:55,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:55,583 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,583 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:55,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:55,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:55,584 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,584 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:55,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:55,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:55,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ef7d521[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:55,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:55,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,589 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:55,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:55,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:55,589 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,589 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:55,591 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:55,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:55,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:55,596 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:55,596 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:55,596 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:55,596 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,596 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:55,601 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,601 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:55,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,601 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:55,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:55,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:55,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:55,602 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:55,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,603 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,606 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:55,606 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:55,606 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:55,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:55,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:55,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:55,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:55,607 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:55,607 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,608 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:55,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:55,708 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:55,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:55,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:55,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:55,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,719 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:55,719 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:55,720 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:55,720 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:55,721 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,721 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,725 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:55,725 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,725 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:55,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:55,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:55,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:55,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:55,727 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:55,727 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:55,727 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:55,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:55,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:55,728 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:55,728 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:55,728 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:55,746 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123118eb450a295e466598521686f982f04c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123118eb450a295e466598521686f982f04c 2023-04-09 07:53:56,154 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/79ed0cba95674d339a033b9d29cec5a3 2023-04-09 07:53:56,161 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/79ed0cba95674d339a033b9d29cec5a3 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3 2023-04-09 07:53:56,167 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3, entries=1, sequenceid=25, filesize=4.9 K 2023-04-09 07:53:56,169 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 441ms, sequenceid=25, compaction requested=true 2023-04-09 07:53:56,169 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:56,169 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:56,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:56,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:56,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:56,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:56,171 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,171 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,171 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,171 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:56,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:56,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@397cfb8a[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:56,173 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:56,173 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,174 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:56,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:56,175 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,175 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:56,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:56,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:56,176 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:56,181 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:56,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:56,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:56,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:56,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:56,185 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,185 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,185 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,186 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,189 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:56,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:56,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:56,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:56,192 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:56,192 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:56,191 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,193 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,193 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,193 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,194 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,194 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:56,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:56,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,292 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:56,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:56,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:56,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:56,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,303 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:56,303 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:56,303 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:56,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:56,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,305 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:56,306 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:56,306 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,306 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:56,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,307 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:56,307 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:56,308 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:56,309 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:56,309 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:56,328 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601013e71a473d00342f28b9ceccad4497275 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601013e71a473d00342f28b9ceccad4497275 2023-04-09 07:53:56,738 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/a9e5e73282754b0596bac73a65e48cfb 2023-04-09 07:53:56,745 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/a9e5e73282754b0596bac73a65e48cfb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb 2023-04-09 07:53:56,751 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb, entries=1, sequenceid=29, filesize=4.9 K 2023-04-09 07:53:56,752 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 443ms, sequenceid=29, compaction requested=true 2023-04-09 07:53:56,752 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:56,752 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:56,753 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:56,753 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:56,753 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,753 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:56,753 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:56,755 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,755 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,756 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,756 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:56,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:56,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7339ba50[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:56,757 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:56,758 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,759 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,760 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:56,760 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:56,760 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,760 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:56,760 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:56,761 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,761 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,761 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,762 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:56,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:56,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:56,764 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:56,765 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:56,765 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:56,765 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,765 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:56,766 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,766 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,766 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,767 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,771 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,773 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:56,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:56,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,774 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,774 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,774 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:56,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:56,775 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,775 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:56,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:56,875 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:56,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:56,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:56,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:56,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,889 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:56,889 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:56,889 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:56,889 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:56,890 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,890 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,892 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:56,892 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:56,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:56,893 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,893 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:56,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:56,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:56,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,894 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:56,894 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:56,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:56,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:56,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:56,895 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:56,895 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:56,895 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:56,926 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c9f1e43696ac4672bae264f7cb57ae2d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103c9f1e43696ac4672bae264f7cb57ae2d 2023-04-09 07:53:56,933 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/8a1ecf0e89984f91bdeacb69d1a4b76c 2023-04-09 07:53:56,939 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/8a1ecf0e89984f91bdeacb69d1a4b76c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c 2023-04-09 07:53:56,945 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c, entries=1, sequenceid=33, filesize=4.9 K 2023-04-09 07:53:56,946 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 51ms, sequenceid=33, compaction requested=true 2023-04-09 07:53:56,946 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:56,946 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:56,947 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:56,948 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,949 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:56,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:56,951 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@653925ee[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:56,951 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:56,951 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,952 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:56,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:56,953 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,953 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:56,958 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:56,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:56,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:56,962 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:56,962 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:56,962 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:56,962 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,962 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:56,963 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,963 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:56,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:56,963 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:56,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:56,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,968 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,968 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:56,968 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:56,968 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:56,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:56,968 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:56,968 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:56,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:56,969 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:56,969 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,969 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,969 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:56,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:56,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:56,970 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:56,970 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:57,070 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:57,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:57,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:57,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:57,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,080 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:57,080 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:57,081 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:57,081 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:57,081 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,081 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,082 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,082 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:57,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:57,083 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,083 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:57,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:57,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,084 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:57,084 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:57,085 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:57,085 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:57,085 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:57,085 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:57,533 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f03049ccb0424cdcbfba0dd20b72706d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907f03049ccb0424cdcbfba0dd20b72706d 2023-04-09 07:53:57,547 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/02b9a2d6e0c640eeb13ba0b85696a88f 2023-04-09 07:53:57,564 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/02b9a2d6e0c640eeb13ba0b85696a88f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f 2023-04-09 07:53:57,587 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f, entries=1, sequenceid=37, filesize=4.9 K 2023-04-09 07:53:57,588 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 503ms, sequenceid=37, compaction requested=true 2023-04-09 07:53:57,588 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:57,588 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:57,588 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:57,588 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:57,588 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:57,589 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:57,624 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:57,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:57,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:57,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,625 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:57,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:57,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:57,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:57,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55d7aab5[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:57,626 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:57,627 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,685 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,685 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:57,686 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:57,686 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:57,685 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,686 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:57,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:57,724 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:57,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:57,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:57,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:57,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:57,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:57,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:57,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:57,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:57,728 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:57,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:57,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:57,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,728 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:57,816 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,816 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:57,816 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:57,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,817 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:57,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:57,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:57,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,818 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:57,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:57,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:57,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:57,862 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:57,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:53:57,862 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:57,863 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:57,863 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,864 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:57,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:57,864 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,864 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,864 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,964 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:57,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:57,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:53:57,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:53:57,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,975 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:57,975 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:57,976 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:53:57,976 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:53:57,977 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,977 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,978 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:57,978 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:57,979 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:57,979 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:53:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:53:57,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:53:57,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:57,980 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:53:57,980 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:53:57,982 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:53:57,982 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:53:57,982 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:53:57,982 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:57,982 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. started... 2023-04-09 07:53:57,982 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fced19ac59ac2562418987229fbaf2d6 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:53:58,002 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112071b1913e9b2c42caad4e0a50ba0ec87f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112071b1913e9b2c42caad4e0a50ba0ec87f 2023-04-09 07:53:58,013 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/44294d1808ee4560ba4dce51e11d126c 2023-04-09 07:53:58,021 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/44294d1808ee4560ba4dce51e11d126c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c 2023-04-09 07:53:58,029 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c, entries=1, sequenceid=41, filesize=4.9 K 2023-04-09 07:53:58,030 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fced19ac59ac2562418987229fbaf2d6 in 48ms, sequenceid=41, compaction requested=true 2023-04-09 07:53:58,030 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:58,030 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:58,030 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:53:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:53:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:53:58,031 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:53:58,032 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,033 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:53:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:58,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:53:58,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ba9bb2e[Count = 0] remaining members to acquire global barrier 2023-04-09 07:53:58,037 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:53:58,037 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,038 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,038 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,039 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:53:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:53:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:53:58,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:53:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:53:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:58,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:53:58,045 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:58,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:58,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:53:58,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:53:58,050 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:53:58,050 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:53:58,050 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:53:58,050 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,050 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:53:58,052 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:53:58,053 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,053 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:58,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:53:58,054 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:53:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,055 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:53:58,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:53:58,064 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:53:58,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:53:58,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:53:58,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-04-09 07:53:58,068 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:53:58,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,068 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:53:58,068 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,068 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,068 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:53:58,069 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:53:58,069 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:53:58,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:53:58,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:53:58,169 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:53:58,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:53:58,172 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-04-09 07:53:58,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:53:58,175 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:53:58,176 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:58,176 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027438176 2023-04-09 07:53:58,177 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:53:58,178 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-04-09 07:53:58,179 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:58,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x33061325 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:53:58,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@748536b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:53:58,198 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:58,660 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103df2e043e08a44d52a9209e30426341a8 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103df2e043e08a44d52a9209e30426341a8 2023-04-09 07:53:58,661 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:58,680 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:53:58,681 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59164, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:53:58,689 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:53:58,690 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34266, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:53:58,710 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/860fc100fd814578baf596e471ba5064 first=Optional[r5] last=Optional[r6] 2023-04-09 07:53:58,711 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/860fc100fd814578baf596e471ba5064}] 2023-04-09 07:53:58,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/860fc100fd814578baf596e471ba5064 for inclusion in fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:58,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-04-09 07:53:58,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:58,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:58,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/860fc100fd814578baf596e471ba5064 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vsehrbfecrth0df2ffn05gn5d0o46c5t8hcp96200lbg2fi9uo1k796d9u9dsuh/family1/860fc100fd814578baf596e471ba5064 2023-04-09 07:53:58,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vsehrbfecrth0df2ffn05gn5d0o46c5t8hcp96200lbg2fi9uo1k796d9u9dsuh/family1/860fc100fd814578baf596e471ba5064 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_ 2023-04-09 07:53:58,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vsehrbfecrth0df2ffn05gn5d0o46c5t8hcp96200lbg2fi9uo1k796d9u9dsuh/family1/860fc100fd814578baf596e471ba5064 into fced19ac59ac2562418987229fbaf2d6/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_ - updating store file list. 2023-04-09 07:53:58,731 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_ into fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:58,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vsehrbfecrth0df2ffn05gn5d0o46c5t8hcp96200lbg2fi9uo1k796d9u9dsuh/family1/860fc100fd814578baf596e471ba5064 into fced19ac59ac2562418987229fbaf2d6/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_) 2023-04-09 07:53:58,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__vsehrbfecrth0df2ffn05gn5d0o46c5t8hcp96200lbg2fi9uo1k796d9u9dsuh/family1/860fc100fd814578baf596e471ba5064 2023-04-09 07:53:58,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:53:58,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:53:58,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): fced19ac59ac2562418987229fbaf2d6/family1 is initiating minor compaction 2023-04-09 07:53:58,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@21c59816; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:58,738 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fced19ac59ac2562418987229fbaf2d6/family1 in testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:58,738 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp, totalSize=49.7 K 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting df2412df21584b3eb1962b4c23740cb0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 91f32b16a0394f6fb2e7b1ae474a7734, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting bd0dd891df7f4c58a4cd21d015e73c74, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting c2bef33e103445568766c8d9f710528c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 79ed0cba95674d339a033b9d29cec5a3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting a9e5e73282754b0596bac73a65e48cfb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:53:58,738 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 8a1ecf0e89984f91bdeacb69d1a4b76c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:53:58,739 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 02b9a2d6e0c640eeb13ba0b85696a88f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:53:58,739 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 44294d1808ee4560ba4dce51e11d126c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:53:58,739 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:53:58,740 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:58,741 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:58,745 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601013e71a473d00342f28b9ceccad4497275 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601013e71a473d00342f28b9ceccad4497275 2023-04-09 07:53:58,747 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103c9f1e43696ac4672bae264f7cb57ae2d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103c9f1e43696ac4672bae264f7cb57ae2d 2023-04-09 07:53:58,747 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-04-09 07:53:58,747 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-04-09 07:53:58,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:58,748 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:58,801 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): fced19ac59ac2562418987229fbaf2d6#family1#compaction#22 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:53:58,822 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/524a3b46ce8747c9a76cb757697b1c7b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/524a3b46ce8747c9a76cb757697b1c7b 2023-04-09 07:53:58,831 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fced19ac59ac2562418987229fbaf2d6/family1 of fced19ac59ac2562418987229fbaf2d6 into 524a3b46ce8747c9a76cb757697b1c7b(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:53:58,831 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:58,831 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1681026838737; duration=0sec 2023-04-09 07:53:58,832 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:58,839 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231460cea0e2f3844f3a29616f67986afdd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231460cea0e2f3844f3a29616f67986afdd 2023-04-09 07:53:58,840 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:58,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_] to archive 2023-04-09 07:53:58,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:58,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/df2412df21584b3eb1962b4c23740cb0 2023-04-09 07:53:58,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91f32b16a0394f6fb2e7b1ae474a7734 2023-04-09 07:53:58,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/bd0dd891df7f4c58a4cd21d015e73c74 2023-04-09 07:53:58,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/c2bef33e103445568766c8d9f710528c 2023-04-09 07:53:58,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/79ed0cba95674d339a033b9d29cec5a3 2023-04-09 07:53:58,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/a9e5e73282754b0596bac73a65e48cfb 2023-04-09 07:53:58,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/8a1ecf0e89984f91bdeacb69d1a4b76c 2023-04-09 07:53:58,868 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/02b9a2d6e0c640eeb13ba0b85696a88f 2023-04-09 07:53:58,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/44294d1808ee4560ba4dce51e11d126c 2023-04-09 07:53:58,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/5c89e784ba1e42e5ba21b09049930dc5_SeqId_44_ 2023-04-09 07:53:58,889 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9495d0942c62451fad37040f72f28f07 first=Optional[r1] last=Optional[r4] 2023-04-09 07:53:58,890 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9495d0942c62451fad37040f72f28f07}] 2023-04-09 07:53:58,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9495d0942c62451fad37040f72f28f07 for inclusion in fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:58,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-04-09 07:53:58,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:58,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:58,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9495d0942c62451fad37040f72f28f07 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f9n9b1aosmq56qv2sondre7q39um92du2ebjcjsiplqtsktdtmdg9q1gqhj2rrvv/family1/9495d0942c62451fad37040f72f28f07 2023-04-09 07:53:58,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f9n9b1aosmq56qv2sondre7q39um92du2ebjcjsiplqtsktdtmdg9q1gqhj2rrvv/family1/9495d0942c62451fad37040f72f28f07 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_ 2023-04-09 07:53:58,905 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f9n9b1aosmq56qv2sondre7q39um92du2ebjcjsiplqtsktdtmdg9q1gqhj2rrvv/family1/9495d0942c62451fad37040f72f28f07 into fced19ac59ac2562418987229fbaf2d6/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_ - updating store file list. 2023-04-09 07:53:58,911 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_ into fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:58,911 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f9n9b1aosmq56qv2sondre7q39um92du2ebjcjsiplqtsktdtmdg9q1gqhj2rrvv/family1/9495d0942c62451fad37040f72f28f07 into fced19ac59ac2562418987229fbaf2d6/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_) 2023-04-09 07:53:58,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__f9n9b1aosmq56qv2sondre7q39um92du2ebjcjsiplqtsktdtmdg9q1gqhj2rrvv/family1/9495d0942c62451fad37040f72f28f07 2023-04-09 07:53:58,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:53:58,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:53:58,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): fced19ac59ac2562418987229fbaf2d6/family1 is initiating minor compaction (all files) 2023-04-09 07:53:58,914 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fced19ac59ac2562418987229fbaf2d6/family1 in testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:53:58,914 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/524a3b46ce8747c9a76cb757697b1c7b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp, totalSize=178.1 K 2023-04-09 07:53:58,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@416c9f25; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:58,915 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting db18950f4ef9465f93763da6fbef6f39, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:53:58,915 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 524a3b46ce8747c9a76cb757697b1c7b, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:53:58,916 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 9b6eecbe7da1446189143d2b7bdea953_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240049 2023-04-09 07:53:58,921 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:58,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:58,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201af30bcf0258f4827a0a9c65ac39e9a1e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201af30bcf0258f4827a0a9c65ac39e9a1e 2023-04-09 07:53:58,927 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120594082eccca4445ecbd2f387190ab984b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120594082eccca4445ecbd2f387190ab984b 2023-04-09 07:53:58,929 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5cd511daad142498ee079d9328fa61c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5cd511daad142498ee079d9328fa61c 2023-04-09 07:53:58,930 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123118eb450a295e466598521686f982f04c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123118eb450a295e466598521686f982f04c 2023-04-09 07:53:58,930 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-04-09 07:53:58,930 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-04-09 07:53:58,931 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:58,931 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:58,992 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): fced19ac59ac2562418987229fbaf2d6#family1#compaction#23 average throughput is 3.99 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:53:59,434 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c6c05f7bc0d8462cab6a96a86684446c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130c6c05f7bc0d8462cab6a96a86684446c 2023-04-09 07:53:59,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:59,472 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1b3c92c360794c3f9f669d3a20a12630 first=Optional[r00] last=Optional[r8] 2023-04-09 07:53:59,473 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1b3c92c360794c3f9f669d3a20a12630}] 2023-04-09 07:53:59,476 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1b3c92c360794c3f9f669d3a20a12630 for inclusion in fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:59,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-04-09 07:53:59,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:53:59,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:59,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1b3c92c360794c3f9f669d3a20a12630 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__cddftfqsau0j1hk8t82fir67r75crvf87tdjld1oaui9uolusavqmbafbf94gt92/family1/1b3c92c360794c3f9f669d3a20a12630 2023-04-09 07:53:59,486 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/.tmp/family1/91e252a3d6264de5aa410c2db2a11890 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/91e252a3d6264de5aa410c2db2a11890 2023-04-09 07:53:59,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__cddftfqsau0j1hk8t82fir67r75crvf87tdjld1oaui9uolusavqmbafbf94gt92/family1/1b3c92c360794c3f9f669d3a20a12630 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/026707d77bc04fcf981cebd1265106a2_SeqId_49_ 2023-04-09 07:53:59,492 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__cddftfqsau0j1hk8t82fir67r75crvf87tdjld1oaui9uolusavqmbafbf94gt92/family1/1b3c92c360794c3f9f669d3a20a12630 into fced19ac59ac2562418987229fbaf2d6/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/026707d77bc04fcf981cebd1265106a2_SeqId_49_ - updating store file list. 2023-04-09 07:53:59,499 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fced19ac59ac2562418987229fbaf2d6/family1 of fced19ac59ac2562418987229fbaf2d6 into 91e252a3d6264de5aa410c2db2a11890(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:53:59,500 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:53:59,500 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/026707d77bc04fcf981cebd1265106a2_SeqId_49_ into fced19ac59ac2562418987229fbaf2d6/family1 2023-04-09 07:53:59,500 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1681026838914; duration=0sec 2023-04-09 07:53:59,500 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__cddftfqsau0j1hk8t82fir67r75crvf87tdjld1oaui9uolusavqmbafbf94gt92/family1/1b3c92c360794c3f9f669d3a20a12630 into fced19ac59ac2562418987229fbaf2d6/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/026707d77bc04fcf981cebd1265106a2_SeqId_49_) 2023-04-09 07:53:59,500 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:53:59,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithMonthlyPolicy__cddftfqsau0j1hk8t82fir67r75crvf87tdjld1oaui9uolusavqmbafbf94gt92/family1/1b3c92c360794c3f9f669d3a20a12630 2023-04-09 07:53:59,503 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:59,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:59,505 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112071b1913e9b2c42caad4e0a50ba0ec87f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112071b1913e9b2c42caad4e0a50ba0ec87f 2023-04-09 07:53:59,506 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128da57c6949d524625b184f12f19a2a549 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128da57c6949d524625b184f12f19a2a549 2023-04-09 07:53:59,508 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113009fb2ef4499b4d0b888c2ef8da9ad07f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113009fb2ef4499b4d0b888c2ef8da9ad07f 2023-04-09 07:53:59,508 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-04-09 07:53:59,508 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-09 07:53:59,509 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:59,509 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:59,509 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:53:59,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:53:59,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:53:59,511 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:53:59,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.3380 sec 2023-04-09 07:53:59,547 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/524a3b46ce8747c9a76cb757697b1c7b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_] to archive 2023-04-09 07:53:59,548 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:53:59,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/db18950f4ef9465f93763da6fbef6f39 2023-04-09 07:53:59,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/524a3b46ce8747c9a76cb757697b1c7b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/524a3b46ce8747c9a76cb757697b1c7b 2023-04-09 07:53:59,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1/9b6eecbe7da1446189143d2b7bdea953_SeqId_47_ 2023-04-09 07:53:59,565 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 17 more 2023-04-09 07:53:59,678 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=259 (was 255) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x42bdd834-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41470 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41522 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38062 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x33061325 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/347772476.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1031621008_15 at /127.0.0.1:41526 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41544 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x33061325-SendThread(127.0.0.1:57867) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41424 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41382 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41378 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41500 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41352 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:57867@0x33061325-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x42bdd834-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41474 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41440 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41390 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1c2320a1-shared-pool-22 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1c2320a1-shared-pool-21 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:41576 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1031621008_15 at /127.0.0.1:41456 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1c2320a1-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38074 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2143285460_15 at /127.0.0.1:38274 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=620 (was 601) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=374 (was 331) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1827 (was 1994) 2023-04-09 07:53:59,695 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=259, OpenFileDescriptor=620, MaxFileDescriptor=60000, SystemLoadAverage=374, ProcessCount=169, AvailableMemoryMB=1825 2023-04-09 07:53:59,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:53:59,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-04-09 07:53:59,704 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:53:59,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-04-09 07:53:59,705 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:53:59,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-09 07:53:59,710 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:53:59,712 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1 empty. 2023-04-09 07:53:59,714 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:53:59,714 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-04-09 07:53:59,741 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-09 07:53:59,743 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => fe2228553ec53030a5c3900b944f37d1, NAME => 'testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:53:59,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-09 07:54:00,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing fe2228553ec53030a5c3900b944f37d1, disabling compactions & flushes 2023-04-09 07:54:00,155 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. after waiting 0 ms 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,155 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,155 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:00,158 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:00,159 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026840159"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026840159"}]},"ts":"1681026840159"} 2023-04-09 07:54:00,161 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:54:00,162 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:00,163 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026840162"}]},"ts":"1681026840162"} 2023-04-09 07:54:00,164 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-04-09 07:54:00,174 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=fe2228553ec53030a5c3900b944f37d1, ASSIGN}] 2023-04-09 07:54:00,176 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=fe2228553ec53030a5c3900b944f37d1, ASSIGN 2023-04-09 07:54:00,177 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=fe2228553ec53030a5c3900b944f37d1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:00,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-09 07:54:00,328 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=fe2228553ec53030a5c3900b944f37d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:00,329 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026840328"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026840328"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026840328"}]},"ts":"1681026840328"} 2023-04-09 07:54:00,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure fe2228553ec53030a5c3900b944f37d1, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:00,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fe2228553ec53030a5c3900b944f37d1, NAME => 'testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:00,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:00,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,491 INFO [StoreOpener-fe2228553ec53030a5c3900b944f37d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,493 DEBUG [StoreOpener-fe2228553ec53030a5c3900b944f37d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:00,493 DEBUG [StoreOpener-fe2228553ec53030a5c3900b944f37d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:00,493 INFO [StoreOpener-fe2228553ec53030a5c3900b944f37d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe2228553ec53030a5c3900b944f37d1 columnFamilyName family1 2023-04-09 07:54:00,495 INFO [StoreOpener-fe2228553ec53030a5c3900b944f37d1-1] regionserver.HStore(310): Store=fe2228553ec53030a5c3900b944f37d1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:00,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:54:00,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:00,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fe2228553ec53030a5c3900b944f37d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58767402, jitterRate=-0.12429746985435486}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:00,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:00,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1., pid=25, masterSystemTime=1681026840485 2023-04-09 07:54:00,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,507 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=fe2228553ec53030a5c3900b944f37d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:00,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026840507"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026840507"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026840507"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026840507"}]},"ts":"1681026840507"} 2023-04-09 07:54:00,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-04-09 07:54:00,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure fe2228553ec53030a5c3900b944f37d1, server=jenkins-hbase4.apache.org,34907,1681026816245 in 178 msec 2023-04-09 07:54:00,514 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-04-09 07:54:00,514 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=fe2228553ec53030a5c3900b944f37d1, ASSIGN in 338 msec 2023-04-09 07:54:00,515 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:00,515 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026840515"}]},"ts":"1681026840515"} 2023-04-09 07:54:00,517 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-04-09 07:54:00,519 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:00,521 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 822 msec 2023-04-09 07:54:00,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-04-09 07:54:00,812 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-04-09 07:54:00,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:00,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,848 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:00,848 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:00,848 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:00,848 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:00,849 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,849 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,851 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:00,851 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:00,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:00,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:00,851 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:00,851 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:00,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:00,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:00,852 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:00,853 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:00,853 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:00,853 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:00,853 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:00,853 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:00,853 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:00,854 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:54:01,279 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ae8ff82a677740518315679f69434bf0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130ae8ff82a677740518315679f69434bf0 2023-04-09 07:54:01,283 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/18763a9c5b2e411fa65aef40afa429fc 2023-04-09 07:54:01,289 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/18763a9c5b2e411fa65aef40afa429fc as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/18763a9c5b2e411fa65aef40afa429fc 2023-04-09 07:54:01,294 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/18763a9c5b2e411fa65aef40afa429fc, entries=1000, sequenceid=5, filesize=165.1 K 2023-04-09 07:54:01,295 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 442ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:01,296 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:01,297 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:01,297 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:01,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:01,299 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:01,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35fc6b31[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:01,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:01,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,302 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:01,302 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,303 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:01,305 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,305 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:01,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:01,308 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:01,308 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:01,308 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:01,308 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,308 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:01,311 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,311 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:01,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,311 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:01,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,312 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,313 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,316 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:01,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:01,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:01,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,316 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:01,317 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:01,317 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:01,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:01,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:01,417 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:01,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:01,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:01,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:01,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,425 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:01,425 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:01,426 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:01,426 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:01,426 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,426 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,427 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,427 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:01,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:01,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:01,428 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,428 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:01,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:01,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:01,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,429 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:01,429 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:01,429 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:01,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:01,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:01,430 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:01,430 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:01,430 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:54:01,459 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128fa8ce96da1bf4a9eb9bd2fc3afdbfa1d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128fa8ce96da1bf4a9eb9bd2fc3afdbfa1d 2023-04-09 07:54:01,870 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/5eccbaa0165842b9aaeb99b3c2a3163e 2023-04-09 07:54:01,876 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/5eccbaa0165842b9aaeb99b3c2a3163e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e 2023-04-09 07:54:01,881 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:54:01,883 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 453ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:01,884 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:01,884 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:01,884 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:01,884 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:01,884 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,884 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:01,884 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:01,886 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,886 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,886 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,886 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:01,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19463e35[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:01,888 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:01,888 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,890 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:01,891 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:01,891 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,891 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:01,892 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:01,892 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:01,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:01,895 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:01,895 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:01,895 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:01,895 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,895 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:01,897 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,897 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:01,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,897 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:01,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:01,898 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,899 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:01,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,901 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,901 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:01,901 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,902 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:01,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:01,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:01,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,902 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:01,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:01,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:02,002 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:02,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:02,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:02,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:02,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,023 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:02,023 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,023 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:02,023 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:02,024 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,024 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,027 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,027 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:02,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:02,027 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,027 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,029 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,038 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:02,039 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,039 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:02,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:02,043 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:02,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:02,043 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:02,044 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:02,072 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120139bb3e0856e84f1e8477acbeed21ee18 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139bb3e0856e84f1e8477acbeed21ee18 2023-04-09 07:54:02,090 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/348ab0be131d496a8d2a52325e8c48ab 2023-04-09 07:54:02,098 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/348ab0be131d496a8d2a52325e8c48ab as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab 2023-04-09 07:54:02,104 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab, entries=1, sequenceid=13, filesize=4.9 K 2023-04-09 07:54:02,105 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 61ms, sequenceid=13, compaction requested=true 2023-04-09 07:54:02,106 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:02,106 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:02,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:02,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:02,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:02,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:02,108 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,108 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,108 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,108 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:02,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:02,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71acbdaa[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:02,110 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:02,110 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,112 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,112 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:02,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:02,112 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:02,113 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:02,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:02,114 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:02,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:02,118 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:02,119 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:02,119 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:02,119 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,119 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:02,121 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,121 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:02,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:02,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:02,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,123 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,125 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,131 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,131 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,131 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:02,134 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:02,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:02,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:02,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:02,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:02,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:02,134 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:02,135 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:02,235 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:02,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:02,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:02,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:02,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,243 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:02,243 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,243 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:02,243 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:02,244 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,244 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,245 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:02,245 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:02,245 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,246 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:02,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:02,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,247 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:02,247 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,247 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:02,247 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:02,247 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:02,247 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:02,247 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:02,247 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:02,265 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057966234a24d94f22b71a242595a24dda to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057966234a24d94f22b71a242595a24dda 2023-04-09 07:54:02,278 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/427d18c1ddc44076b0951e4b42689c10 2023-04-09 07:54:02,287 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/427d18c1ddc44076b0951e4b42689c10 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10 2023-04-09 07:54:02,296 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10, entries=1, sequenceid=17, filesize=4.9 K 2023-04-09 07:54:02,297 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 50ms, sequenceid=17, compaction requested=true 2023-04-09 07:54:02,297 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:02,297 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:02,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:02,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:02,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:02,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:02,302 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,302 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,303 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,304 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:02,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:02,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@379a087a[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:02,306 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:02,306 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,307 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:02,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:02,308 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:02,308 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:02,311 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:02,312 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,312 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:02,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,312 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:02,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:02,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:02,316 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:02,316 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:02,316 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:02,316 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,317 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:02,318 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,318 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:02,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,318 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:02,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:02,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:02,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:02,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:02,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,319 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,321 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:02,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,324 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:02,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:02,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:02,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:02,324 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:02,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:02,324 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:02,324 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:02,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,325 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,425 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:02,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:02,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:02,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:02,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,435 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:02,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:02,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:02,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,436 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,437 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:02,437 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:02,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:02,437 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:02,437 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:02,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:02,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:02,438 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:02,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:02,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:02,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:02,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:02,439 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:02,439 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:02,439 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:02,858 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512280341b9aa508d438f84085844fad60a4c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280341b9aa508d438f84085844fad60a4c 2023-04-09 07:54:03,264 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/16ca483b91564e6b959fdd7b19102bd8 2023-04-09 07:54:03,270 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/16ca483b91564e6b959fdd7b19102bd8 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8 2023-04-09 07:54:03,274 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8, entries=1, sequenceid=21, filesize=4.9 K 2023-04-09 07:54:03,275 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 836ms, sequenceid=21, compaction requested=true 2023-04-09 07:54:03,275 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:03,275 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:03,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:03,277 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:03,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:03,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:03,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6169485b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:03,279 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:03,279 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,280 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:03,281 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:03,281 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,281 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:03,282 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:03,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:03,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:03,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:03,284 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:03,284 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:03,284 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:03,284 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,284 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:03,287 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,287 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:03,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,287 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:03,287 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,288 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:03,290 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:03,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:03,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:03,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:03,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:03,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:03,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:03,291 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:03,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:03,391 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:03,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:03,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:03,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:03,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,399 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:03,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:03,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:03,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:03,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,401 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:03,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:03,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:03,401 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:03,401 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:03,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:03,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:03,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:03,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:03,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:03,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:03,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:03,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:03,402 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:03,403 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:03,403 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:03,819 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b8574444fdae453e889da624291e6836 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b8574444fdae453e889da624291e6836 2023-04-09 07:54:04,225 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/a52cd9860ec64e5f843a4adfe7038c24 2023-04-09 07:54:04,230 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/a52cd9860ec64e5f843a4adfe7038c24 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24 2023-04-09 07:54:04,235 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24, entries=1, sequenceid=25, filesize=4.9 K 2023-04-09 07:54:04,236 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 833ms, sequenceid=25, compaction requested=true 2023-04-09 07:54:04,236 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:04,236 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:04,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:04,238 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:04,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:04,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49f665d5[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:04,239 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:04,240 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:04,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:04,242 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,242 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:04,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:04,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:04,243 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:04,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:04,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:04,246 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:04,246 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:04,246 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:04,246 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,246 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:04,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:04,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:04,251 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:04,252 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:04,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:04,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:04,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:04,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:04,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,256 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:04,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:04,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:04,256 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:04,356 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:04,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:04,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:04,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:04,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,363 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:04,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:04,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:04,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:04,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,365 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:04,365 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:04,365 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:04,365 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:04,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:04,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,366 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,366 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:04,366 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:04,366 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:04,366 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:04,366 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:04,366 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:04,366 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:04,366 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:04,366 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:04,367 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:04,393 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:04,394 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33061325 to 127.0.0.1:57867 2023-04-09 07:54:04,394 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:04,394 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:04,395 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34313c3e to 127.0.0.1:57867 2023-04-09 07:54:04,395 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:04,395 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:04,395 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77d62183 to 127.0.0.1:57867 2023-04-09 07:54:04,395 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:04,799 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101a6662761611d4fe9942b038de5956b34 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101a6662761611d4fe9942b038de5956b34 2023-04-09 07:54:05,205 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/23f5faf97cde4ba5bf9a15bce3c62217 2023-04-09 07:54:05,210 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/23f5faf97cde4ba5bf9a15bce3c62217 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217 2023-04-09 07:54:05,215 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217, entries=1, sequenceid=29, filesize=4.9 K 2023-04-09 07:54:05,216 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 849ms, sequenceid=29, compaction requested=true 2023-04-09 07:54:05,216 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:05,216 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:05,216 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:05,218 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:05,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:05,218 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,218 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:05,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:05,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:05,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:05,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:05,220 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5da3c0ac[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:05,220 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:05,220 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,221 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,221 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:05,221 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:05,221 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,221 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:05,221 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:05,222 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,222 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:05,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:05,222 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:05,223 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:05,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:05,224 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:05,225 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:05,225 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:05,225 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,225 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:05,227 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,227 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:05,227 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:05,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,228 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:05,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:05,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,229 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:05,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:05,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:05,231 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:05,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:05,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:05,231 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,232 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:05,232 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,232 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,332 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:05,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:05,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:05,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:05,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,339 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:05,339 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:05,339 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:05,339 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:05,340 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,340 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,341 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,341 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:05,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:05,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:05,341 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:05,341 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:05,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:05,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:05,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:05,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:05,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:05,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:05,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:05,342 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:05,343 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:05,343 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:05,756 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034a780f2210ad412582cb30c5f039b5dc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034a780f2210ad412582cb30c5f039b5dc 2023-04-09 07:54:06,012 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:06,162 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/b21a946c9b894c00b1b6babbed217c83 2023-04-09 07:54:06,168 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/b21a946c9b894c00b1b6babbed217c83 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83 2023-04-09 07:54:06,172 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83, entries=1, sequenceid=33, filesize=4.9 K 2023-04-09 07:54:06,173 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 830ms, sequenceid=33, compaction requested=true 2023-04-09 07:54:06,173 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:06,173 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:06,173 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:06,173 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:06,173 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,173 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:06,173 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:06,175 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:06,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:06,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:06,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:06,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:06,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a9dcaba[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:06,176 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:06,176 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,178 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,178 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:06,178 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,179 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:06,178 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:06,179 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:06,180 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:06,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:06,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:06,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:06,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:06,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:06,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:06,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:06,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:06,183 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,183 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:06,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,183 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:06,184 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:06,185 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,186 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,186 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:06,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:06,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:06,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:06,187 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:06,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:06,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:06,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:06,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,287 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:06,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:06,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:06,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:06,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,294 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:06,294 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:06,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:06,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:06,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,297 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,297 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:06,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:06,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:06,297 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:06,297 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:06,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:06,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:06,298 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:06,298 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:06,298 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:06,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:06,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:06,299 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:06,299 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:06,299 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:06,712 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a4ab6808bbe94641ae00f667d63b9d8c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907a4ab6808bbe94641ae00f667d63b9d8c 2023-04-09 07:54:07,117 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/403cdeeb52854782999dff64775a5e83 2023-04-09 07:54:07,122 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/403cdeeb52854782999dff64775a5e83 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83 2023-04-09 07:54:07,126 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83, entries=1, sequenceid=37, filesize=4.9 K 2023-04-09 07:54:07,127 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 828ms, sequenceid=37, compaction requested=true 2023-04-09 07:54:07,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:07,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:07,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:07,130 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:07,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:07,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:07,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:07,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:07,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:07,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:07,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@119d6622[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:07,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:07,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,137 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,137 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:07,137 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,137 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:07,137 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:07,137 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:07,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:07,139 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:07,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:07,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:07,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:07,142 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:07,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:07,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:07,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,142 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:07,143 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,143 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,143 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:07,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:07,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:07,144 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:07,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:07,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:07,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:07,147 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:07,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:07,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,147 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:07,248 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:07,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:07,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:07,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:07,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,255 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:07,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:07,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:07,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:07,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,256 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,257 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:07,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:07,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:07,257 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,257 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:07,257 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:07,257 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:07,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,258 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:07,258 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:07,258 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:07,258 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:07,258 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:07,258 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:07,258 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:07,258 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:07,258 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. started... 2023-04-09 07:54:07,259 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing fe2228553ec53030a5c3900b944f37d1 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:07,679 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511208b8a4a9c4afa496bbe4514f052483a49 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511208b8a4a9c4afa496bbe4514f052483a49 2023-04-09 07:54:08,085 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/5c63e5365d774f12902e67a7ca3603f4 2023-04-09 07:54:08,090 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/5c63e5365d774f12902e67a7ca3603f4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4 2023-04-09 07:54:08,095 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4, entries=1, sequenceid=41, filesize=4.9 K 2023-04-09 07:54:08,096 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for fe2228553ec53030a5c3900b944f37d1 in 837ms, sequenceid=41, compaction requested=true 2023-04-09 07:54:08,096 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:08,096 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:08,096 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:08,097 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,097 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:08,098 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,098 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:08,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:08,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:08,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bbf6109[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:08,099 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:08,099 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,100 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,100 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:08,100 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:08,101 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,101 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:08,101 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:08,102 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:08,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:08,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:08,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:08,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:08,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:08,104 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:08,104 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:08,104 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:08,104 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,104 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:08,110 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,110 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,110 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:08,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:08,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,111 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:08,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:08,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:08,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:08,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:08,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:08,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-04-09 07:54:08,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:08,114 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:08,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:08,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:08,215 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:08,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:08,217 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-04-09 07:54:08,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:08,219 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:08,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:08,220 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027448220 2023-04-09 07:54:08,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:54:08,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-04-09 07:54:08,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:08,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x22b81f9d to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:08,227 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7fd5bdfa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:08,227 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,068 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cb17b45b51ee491aab0c5f0346ae60bb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103cb17b45b51ee491aab0c5f0346ae60bb 2023-04-09 07:54:09,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:09,081 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:09,082 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47992, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:09,089 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:09,091 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41888, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:09,100 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fa6baa86ad4747e7aff1c3575ec479d4 first=Optional[r3] last=Optional[r6] 2023-04-09 07:54:09,101 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fa6baa86ad4747e7aff1c3575ec479d4}] 2023-04-09 07:54:09,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fa6baa86ad4747e7aff1c3575ec479d4 for inclusion in fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:09,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-09 07:54:09,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:09,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:09,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/fa6baa86ad4747e7aff1c3575ec479d4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rbt3vek1r1iqpop6jvb1rcrtqrhjg27mh96oqjo03l9meoq3ara2vjab91lt7cme/family1/fa6baa86ad4747e7aff1c3575ec479d4 2023-04-09 07:54:09,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rbt3vek1r1iqpop6jvb1rcrtqrhjg27mh96oqjo03l9meoq3ara2vjab91lt7cme/family1/fa6baa86ad4747e7aff1c3575ec479d4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_ 2023-04-09 07:54:09,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rbt3vek1r1iqpop6jvb1rcrtqrhjg27mh96oqjo03l9meoq3ara2vjab91lt7cme/family1/fa6baa86ad4747e7aff1c3575ec479d4 into fe2228553ec53030a5c3900b944f37d1/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_ - updating store file list. 2023-04-09 07:54:09,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_ into fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:09,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rbt3vek1r1iqpop6jvb1rcrtqrhjg27mh96oqjo03l9meoq3ara2vjab91lt7cme/family1/fa6baa86ad4747e7aff1c3575ec479d4 into fe2228553ec53030a5c3900b944f37d1/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_) 2023-04-09 07:54:09,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__rbt3vek1r1iqpop6jvb1rcrtqrhjg27mh96oqjo03l9meoq3ara2vjab91lt7cme/family1/fa6baa86ad4747e7aff1c3575ec479d4 2023-04-09 07:54:09,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:54:09,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:54:09,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): fe2228553ec53030a5c3900b944f37d1/family1 is initiating minor compaction 2023-04-09 07:54:09,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@16fb1562; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:09,120 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fe2228553ec53030a5c3900b944f37d1/family1 in testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:54:09,120 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp, totalSize=50.0 K 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5eccbaa0165842b9aaeb99b3c2a3163e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 348ab0be131d496a8d2a52325e8c48ab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 427d18c1ddc44076b0951e4b42689c10, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 16ca483b91564e6b959fdd7b19102bd8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting a52cd9860ec64e5f843a4adfe7038c24, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 23f5faf97cde4ba5bf9a15bce3c62217, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:54:09,120 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting b21a946c9b894c00b1b6babbed217c83, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:54:09,121 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 403cdeeb52854782999dff64775a5e83, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:54:09,121 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5c63e5365d774f12902e67a7ca3603f4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:54:09,121 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 324820e0aced471da3701c4c1919c6f6_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:54:09,122 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:09,123 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:09,124 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280341b9aa508d438f84085844fad60a4c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512280341b9aa508d438f84085844fad60a4c 2023-04-09 07:54:09,126 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b8574444fdae453e889da624291e6836 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231b8574444fdae453e889da624291e6836 2023-04-09 07:54:09,127 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101a6662761611d4fe9942b038de5956b34 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101a6662761611d4fe9942b038de5956b34 2023-04-09 07:54:09,129 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034a780f2210ad412582cb30c5f039b5dc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601034a780f2210ad412582cb30c5f039b5dc 2023-04-09 07:54:09,129 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-04-09 07:54:09,129 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-09 07:54:09,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,130 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,147 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): fe2228553ec53030a5c3900b944f37d1#family1#compaction#34 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:09,561 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056e95b63038994fa7b3cb63ba69b291ca to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512056e95b63038994fa7b3cb63ba69b291ca 2023-04-09 07:54:09,562 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:09,586 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1a1d30c474b04776b997a11ef5985da3 first=Optional[r1] last=Optional[r2] 2023-04-09 07:54:09,586 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1a1d30c474b04776b997a11ef5985da3}] 2023-04-09 07:54:09,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1a1d30c474b04776b997a11ef5985da3 for inclusion in fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:09,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-04-09 07:54:09,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:09,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:09,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1a1d30c474b04776b997a11ef5985da3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g8m507pr1c6gd87633mlaljmdg17q3agteikcu73unpvvgi82ptbit09j1t5si83/family1/1a1d30c474b04776b997a11ef5985da3 2023-04-09 07:54:09,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g8m507pr1c6gd87633mlaljmdg17q3agteikcu73unpvvgi82ptbit09j1t5si83/family1/1a1d30c474b04776b997a11ef5985da3 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/8f10808e474741f0aa999b538d02c815_SeqId_46_ 2023-04-09 07:54:09,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g8m507pr1c6gd87633mlaljmdg17q3agteikcu73unpvvgi82ptbit09j1t5si83/family1/1a1d30c474b04776b997a11ef5985da3 into fe2228553ec53030a5c3900b944f37d1/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/8f10808e474741f0aa999b538d02c815_SeqId_46_ - updating store file list. 2023-04-09 07:54:09,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/8f10808e474741f0aa999b538d02c815_SeqId_46_ into fe2228553ec53030a5c3900b944f37d1/family1 2023-04-09 07:54:09,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g8m507pr1c6gd87633mlaljmdg17q3agteikcu73unpvvgi82ptbit09j1t5si83/family1/1a1d30c474b04776b997a11ef5985da3 into fe2228553ec53030a5c3900b944f37d1/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/8f10808e474741f0aa999b538d02c815_SeqId_46_) 2023-04-09 07:54:09,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithWeeklyPolicy__g8m507pr1c6gd87633mlaljmdg17q3agteikcu73unpvvgi82ptbit09j1t5si83/family1/1a1d30c474b04776b997a11ef5985da3 2023-04-09 07:54:09,602 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:09,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:09,604 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139bb3e0856e84f1e8477acbeed21ee18 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120139bb3e0856e84f1e8477acbeed21ee18 2023-04-09 07:54:09,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057966234a24d94f22b71a242595a24dda to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057966234a24d94f22b71a242595a24dda 2023-04-09 07:54:09,606 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-04-09 07:54:09,606 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-09 07:54:09,606 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:09,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:09,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:09,607 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:09,609 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.3900 sec 2023-04-09 07:54:09,709 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=249 (was 259), OpenFileDescriptor=596 (was 620), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=324 (was 374), ProcessCount=169 (was 169), AvailableMemoryMB=1765 (was 1825) 2023-04-09 07:54:09,723 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=249, OpenFileDescriptor=596, MaxFileDescriptor=60000, SystemLoadAverage=324, ProcessCount=169, AvailableMemoryMB=1764 2023-04-09 07:54:09,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-09 07:54:09,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:09,728 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:09,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-04-09 07:54:09,729 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:09,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:09,731 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:09,731 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:09,732 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:09,732 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462 empty. 2023-04-09 07:54:09,732 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c empty. 2023-04-09 07:54:09,732 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a empty. 2023-04-09 07:54:09,733 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:09,733 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:09,733 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:09,733 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-04-09 07:54:09,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:09,962 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/.tmp/family1/d3cca9671a674fd4af043fc693d4232e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/d3cca9671a674fd4af043fc693d4232e 2023-04-09 07:54:09,967 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in fe2228553ec53030a5c3900b944f37d1/family1 of fe2228553ec53030a5c3900b944f37d1 into d3cca9671a674fd4af043fc693d4232e(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:09,967 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:54:09,968 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1681026849120; duration=0sec 2023-04-09 07:54:09,968 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:10,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:10,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_] to archive 2023-04-09 07:54:10,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:10,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5eccbaa0165842b9aaeb99b3c2a3163e 2023-04-09 07:54:10,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/348ab0be131d496a8d2a52325e8c48ab 2023-04-09 07:54:10,057 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/427d18c1ddc44076b0951e4b42689c10 2023-04-09 07:54:10,058 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/16ca483b91564e6b959fdd7b19102bd8 2023-04-09 07:54:10,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/a52cd9860ec64e5f843a4adfe7038c24 2023-04-09 07:54:10,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/23f5faf97cde4ba5bf9a15bce3c62217 2023-04-09 07:54:10,061 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/b21a946c9b894c00b1b6babbed217c83 2023-04-09 07:54:10,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/403cdeeb52854782999dff64775a5e83 2023-04-09 07:54:10,063 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/5c63e5365d774f12902e67a7ca3603f4 2023-04-09 07:54:10,065 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/family1/324820e0aced471da3701c4c1919c6f6_SeqId_44_ 2023-04-09 07:54:10,142 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:10,143 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => f0431b4fc004fb6549afd7e745a9bb5c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:10,143 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6e9e510b2ac2237db799773119e0e62a, NAME => 'testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:10,143 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 16deafa9a5d9b83ab649c63be2104462, NAME => 'testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:10,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing f0431b4fc004fb6549afd7e745a9bb5c, disabling compactions & flushes 2023-04-09 07:54:10,550 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. after waiting 0 ms 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,550 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,550 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:10,555 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,555 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 16deafa9a5d9b83ab649c63be2104462, disabling compactions & flushes 2023-04-09 07:54:10,555 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. after waiting 0 ms 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,556 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 16deafa9a5d9b83ab649c63be2104462: 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 6e9e510b2ac2237db799773119e0e62a, disabling compactions & flushes 2023-04-09 07:54:10,556 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. after waiting 0 ms 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,556 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,556 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 6e9e510b2ac2237db799773119e0e62a: 2023-04-09 07:54:10,558 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:10,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850559"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026850559"}]},"ts":"1681026850559"} 2023-04-09 07:54:10,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850559"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026850559"}]},"ts":"1681026850559"} 2023-04-09 07:54:10,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1681026850559"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026850559"}]},"ts":"1681026850559"} 2023-04-09 07:54:10,562 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-09 07:54:10,563 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:10,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026850563"}]},"ts":"1681026850563"} 2023-04-09 07:54:10,564 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-04-09 07:54:10,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=16deafa9a5d9b83ab649c63be2104462, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6e9e510b2ac2237db799773119e0e62a, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0431b4fc004fb6549afd7e745a9bb5c, ASSIGN}] 2023-04-09 07:54:10,570 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0431b4fc004fb6549afd7e745a9bb5c, ASSIGN 2023-04-09 07:54:10,570 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6e9e510b2ac2237db799773119e0e62a, ASSIGN 2023-04-09 07:54:10,570 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=16deafa9a5d9b83ab649c63be2104462, ASSIGN 2023-04-09 07:54:10,571 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0431b4fc004fb6549afd7e745a9bb5c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:10,571 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6e9e510b2ac2237db799773119e0e62a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:10,571 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=16deafa9a5d9b83ab649c63be2104462, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:10,723 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=16deafa9a5d9b83ab649c63be2104462, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,723 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f0431b4fc004fb6549afd7e745a9bb5c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,723 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6e9e510b2ac2237db799773119e0e62a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,723 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026850722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026850722"}]},"ts":"1681026850722"} 2023-04-09 07:54:10,723 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850723"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026850723"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026850723"}]},"ts":"1681026850723"} 2023-04-09 07:54:10,723 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1681026850722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026850722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026850722"}]},"ts":"1681026850722"} 2023-04-09 07:54:10,725 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure 16deafa9a5d9b83ab649c63be2104462, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:10,725 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 6e9e510b2ac2237db799773119e0e62a, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:10,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure f0431b4fc004fb6549afd7e745a9bb5c, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:10,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:10,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 16deafa9a5d9b83ab649c63be2104462, NAME => 'testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.', STARTKEY => '', ENDKEY => '1'} 2023-04-09 07:54:10,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,882 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,883 DEBUG [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/family1 2023-04-09 07:54:10,883 DEBUG [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/family1 2023-04-09 07:54:10,884 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 16deafa9a5d9b83ab649c63be2104462 columnFamilyName family1 2023-04-09 07:54:10,886 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] regionserver.HStore(310): Store=16deafa9a5d9b83ab649c63be2104462/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,886 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,887 DEBUG [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/family2 2023-04-09 07:54:10,887 DEBUG [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/family2 2023-04-09 07:54:10,888 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 16deafa9a5d9b83ab649c63be2104462 columnFamilyName family2 2023-04-09 07:54:10,889 INFO [StoreOpener-16deafa9a5d9b83ab649c63be2104462-1] regionserver.HStore(310): Store=16deafa9a5d9b83ab649c63be2104462/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:10,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:54:10,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:10,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 16deafa9a5d9b83ab649c63be2104462; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59066270, jitterRate=-0.119843989610672}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:10,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 16deafa9a5d9b83ab649c63be2104462: 2023-04-09 07:54:10,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462., pid=31, masterSystemTime=1681026850877 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:10,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6e9e510b2ac2237db799773119e0e62a, NAME => 'testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.', STARTKEY => '1', ENDKEY => '2'} 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,898 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=16deafa9a5d9b83ab649c63be2104462, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,898 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850898"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026850898"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026850898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026850898"}]},"ts":"1681026850898"} 2023-04-09 07:54:10,900 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,901 DEBUG [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/family1 2023-04-09 07:54:10,901 DEBUG [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/family1 2023-04-09 07:54:10,901 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e9e510b2ac2237db799773119e0e62a columnFamilyName family1 2023-04-09 07:54:10,902 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-04-09 07:54:10,902 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure 16deafa9a5d9b83ab649c63be2104462, server=jenkins-hbase4.apache.org,34907,1681026816245 in 175 msec 2023-04-09 07:54:10,902 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] regionserver.HStore(310): Store=6e9e510b2ac2237db799773119e0e62a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,903 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=16deafa9a5d9b83ab649c63be2104462, ASSIGN in 334 msec 2023-04-09 07:54:10,904 DEBUG [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/family2 2023-04-09 07:54:10,905 DEBUG [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/family2 2023-04-09 07:54:10,906 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6e9e510b2ac2237db799773119e0e62a columnFamilyName family2 2023-04-09 07:54:10,906 INFO [StoreOpener-6e9e510b2ac2237db799773119e0e62a-1] regionserver.HStore(310): Store=6e9e510b2ac2237db799773119e0e62a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:10,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:54:10,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:10,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6e9e510b2ac2237db799773119e0e62a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59883085, jitterRate=-0.10767249763011932}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:10,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6e9e510b2ac2237db799773119e0e62a: 2023-04-09 07:54:10,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a., pid=32, masterSystemTime=1681026850877 2023-04-09 07:54:10,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:10,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f0431b4fc004fb6549afd7e745a9bb5c, NAME => 'testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.', STARTKEY => '2', ENDKEY => ''} 2023-04-09 07:54:10,914 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6e9e510b2ac2237db799773119e0e62a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,914 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1681026850914"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026850914"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026850914"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026850914"}]},"ts":"1681026850914"} 2023-04-09 07:54:10,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:10,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,916 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,917 DEBUG [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1 2023-04-09 07:54:10,917 DEBUG [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1 2023-04-09 07:54:10,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-04-09 07:54:10,917 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 6e9e510b2ac2237db799773119e0e62a, server=jenkins-hbase4.apache.org,34907,1681026816245 in 190 msec 2023-04-09 07:54:10,917 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0431b4fc004fb6549afd7e745a9bb5c columnFamilyName family1 2023-04-09 07:54:10,918 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] regionserver.HStore(310): Store=f0431b4fc004fb6549afd7e745a9bb5c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,918 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6e9e510b2ac2237db799773119e0e62a, ASSIGN in 349 msec 2023-04-09 07:54:10,919 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,920 DEBUG [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family2 2023-04-09 07:54:10,920 DEBUG [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family2 2023-04-09 07:54:10,920 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f0431b4fc004fb6549afd7e745a9bb5c columnFamilyName family2 2023-04-09 07:54:10,921 INFO [StoreOpener-f0431b4fc004fb6549afd7e745a9bb5c-1] regionserver.HStore(310): Store=f0431b4fc004fb6549afd7e745a9bb5c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:10,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:10,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:54:10,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:10,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f0431b4fc004fb6549afd7e745a9bb5c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60632894, jitterRate=-0.0964994728565216}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:10,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:10,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c., pid=33, masterSystemTime=1681026850877 2023-04-09 07:54:10,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:10,928 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f0431b4fc004fb6549afd7e745a9bb5c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:10,929 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1681026850928"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026850928"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026850928"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026850928"}]},"ts":"1681026850928"} 2023-04-09 07:54:10,931 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=30 2023-04-09 07:54:10,931 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure f0431b4fc004fb6549afd7e745a9bb5c, server=jenkins-hbase4.apache.org,34907,1681026816245 in 204 msec 2023-04-09 07:54:10,933 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-04-09 07:54:10,933 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f0431b4fc004fb6549afd7e745a9bb5c, ASSIGN in 363 msec 2023-04-09 07:54:10,934 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:10,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026850934"}]},"ts":"1681026850934"} 2023-04-09 07:54:10,935 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-04-09 07:54:10,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:10,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2120 sec 2023-04-09 07:54:11,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-04-09 07:54:11,835 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-04-09 07:54:11,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:11,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,844 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-04-09 07:54:11,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:11,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-04-09 07:54:11,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-04-09 07:54:11,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:11,845 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:11,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:11,846 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:11,846 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,846 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:11,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-04-09 07:54:11,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:11,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-04-09 07:54:11,847 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:11,847 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. started... 2023-04-09 07:54:11,847 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:11,847 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. started... 2023-04-09 07:54:11,848 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:11,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:54:11,848 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f0431b4fc004fb6549afd7e745a9bb5c 2/2 column families, dataSize=48 B heapSize=632 B 2023-04-09 07:54:11,848 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:11,848 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. started... 2023-04-09 07:54:11,848 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e9e510b2ac2237db799773119e0e62a: 2023-04-09 07:54:11,848 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:11,849 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 16deafa9a5d9b83ab649c63be2104462: 2023-04-09 07:54:11,849 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:54:11,849 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:11,849 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:54:12,261 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ce8110f07af445fe87e27b5da55bf43e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce8110f07af445fe87e27b5da55bf43e 2023-04-09 07:54:12,667 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp/family1/893b8fbfe56e4df58ade9c049dfbe9df 2023-04-09 07:54:12,672 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp/family1/893b8fbfe56e4df58ade9c049dfbe9df as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df 2023-04-09 07:54:12,678 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df, entries=1, sequenceid=5, filesize=4.9 K 2023-04-09 07:54:12,679 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for f0431b4fc004fb6549afd7e745a9bb5c in 831ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:12,679 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-04-09 07:54:12,680 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:12,680 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:54:12,681 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:54:12,681 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,681 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-04-09 07:54:12,681 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-04-09 07:54:12,682 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,682 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-04-09 07:54:12,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@285a6a72[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:12,684 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-04-09 07:54:12,685 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,689 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,689 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-04-09 07:54:12,689 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-04-09 07:54:12,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,690 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:12,689 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:12,691 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-04-09 07:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:12,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:12,691 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-04-09 07:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:12,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:12,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-04-09 07:54:12,694 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-04-09 07:54:12,694 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:12,694 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:12,694 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,694 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:12,696 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,696 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,696 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:12,696 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,697 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:12,703 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:12,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:12,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-04-09 07:54:12,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:12,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:12,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:12,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:12,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,704 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:12,804 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:12,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:12,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,811 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-04-09 07:54:12,811 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:12,811 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-04-09 07:54:12,811 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-04-09 07:54:12,812 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,812 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,813 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:12,813 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:12,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:12,814 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:12,814 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:12,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:12,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,814 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-04-09 07:54:12,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. started... 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. started... 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6e9e510b2ac2237db799773119e0e62a: 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:54:12,815 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:12,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:54:12,815 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing f0431b4fc004fb6549afd7e745a9bb5c 2/2 column families, dataSize=90 B heapSize=752 B 2023-04-09 07:54:12,816 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. started... 2023-04-09 07:54:12,816 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 16deafa9a5d9b83ab649c63be2104462: 2023-04-09 07:54:12,816 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:54:12,816 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:54:13,229 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409cfd9e939a2444504bcc5cf12693349f2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409cfd9e939a2444504bcc5cf12693349f2 2023-04-09 07:54:13,635 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp/family1/55398a0504a2413b9ac4b68266ebc781 2023-04-09 07:54:13,640 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp/family1/55398a0504a2413b9ac4b68266ebc781 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781 2023-04-09 07:54:13,644 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781, entries=2, sequenceid=9, filesize=5.0 K 2023-04-09 07:54:13,645 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for f0431b4fc004fb6549afd7e745a9bb5c in 830ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:13,646 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:13,646 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:13,646 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:54:13,646 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:54:13,646 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-04-09 07:54:13,647 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-04-09 07:54:13,648 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,648 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:13,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:13,649 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,649 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-04-09 07:54:13,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cc6bdd9[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:13,650 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-04-09 07:54:13,650 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,651 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-04-09 07:54:13,652 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-04-09 07:54:13,652 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:13,652 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:13,653 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-04-09 07:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:13,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:13,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:13,653 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-04-09 07:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:13,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-04-09 07:54:13,655 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-04-09 07:54:13,655 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:13,655 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:13,656 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,656 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:13,658 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,658 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:13,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,658 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:13,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,660 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:13,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,661 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:13,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:13,663 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:13,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-04-09 07:54:13,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:13,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-04-09 07:54:13,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:13,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:13,664 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,664 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,664 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,664 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:13,664 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:13,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:13,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:13,764 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:13,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:13,771 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-04-09 07:54:13,773 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-09 07:54:13,774 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-09 07:54:13,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:13,774 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027453774 2023-04-09 07:54:13,775 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:13,775 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-04-09 07:54:13,775 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-09 07:54:13,776 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x35fecb8d to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:13,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67968aa5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:13,780 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230409 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-04-09 07:54:14,608 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409bc702e6ce02846d382f0468433421ce9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409bc702e6ce02846d382f0468433421ce9 2023-04-09 07:54:14,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-09 07:54:14,622 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:14,623 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33156, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:14,631 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:14,633 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60276, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:14,642 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230409/family1/f43bd299365244d7a075bdb8699f5cfc first=Optional[k0] last=Optional[k1] 2023-04-09 07:54:14,643 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230409/family1/f43bd299365244d7a075bdb8699f5cfc}] 2023-04-09 07:54:14,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230409/family1/f43bd299365244d7a075bdb8699f5cfc for inclusion in f0431b4fc004fb6549afd7e745a9bb5c/family1 2023-04-09 07:54:14,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-04-09 07:54:14,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-09 07:54:14,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:14,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230409/family1/f43bd299365244d7a075bdb8699f5cfc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testScannerOnBulkLoadRefHFiles__d6tjhmth2ia8her4hh1m105r87dl1sinb1kbhqssobc9b306uhnpmtr0u5abo13d/family1/f43bd299365244d7a075bdb8699f5cfc 2023-04-09 07:54:14,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testScannerOnBulkLoadRefHFiles__d6tjhmth2ia8her4hh1m105r87dl1sinb1kbhqssobc9b306uhnpmtr0u5abo13d/family1/f43bd299365244d7a075bdb8699f5cfc as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_ 2023-04-09 07:54:14,655 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testScannerOnBulkLoadRefHFiles__d6tjhmth2ia8her4hh1m105r87dl1sinb1kbhqssobc9b306uhnpmtr0u5abo13d/family1/f43bd299365244d7a075bdb8699f5cfc into f0431b4fc004fb6549afd7e745a9bb5c/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_ - updating store file list. 2023-04-09 07:54:14,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_ into f0431b4fc004fb6549afd7e745a9bb5c/family1 2023-04-09 07:54:14,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testScannerOnBulkLoadRefHFiles__d6tjhmth2ia8her4hh1m105r87dl1sinb1kbhqssobc9b306uhnpmtr0u5abo13d/family1/f43bd299365244d7a075bdb8699f5cfc into f0431b4fc004fb6549afd7e745a9bb5c/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_) 2023-04-09 07:54:14,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testScannerOnBulkLoadRefHFiles__d6tjhmth2ia8her4hh1m105r87dl1sinb1kbhqssobc9b306uhnpmtr0u5abo13d/family1/f43bd299365244d7a075bdb8699f5cfc 2023-04-09 07:54:14,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:54:14,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:54:14,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): f0431b4fc004fb6549afd7e745a9bb5c/family1 is initiating minor compaction (all files) 2023-04-09 07:54:14,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@76d21118; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:14,661 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f0431b4fc004fb6549afd7e745a9bb5c/family1 in testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:54:14,661 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp, totalSize=15.1 K 2023-04-09 07:54:14,662 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 893b8fbfe56e4df58ade9c049dfbe9df, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026851835 2023-04-09 07:54:14,662 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 55398a0504a2413b9ac4b68266ebc781, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026851835 2023-04-09 07:54:14,663 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting e3588d437e8848dbbde71f724d29b08c_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026851835 2023-04-09 07:54:14,663 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-09 07:54:14,664 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:14,665 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce8110f07af445fe87e27b5da55bf43e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce8110f07af445fe87e27b5da55bf43e 2023-04-09 07:54:14,666 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409cfd9e939a2444504bcc5cf12693349f2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230409cfd9e939a2444504bcc5cf12693349f2 2023-04-09 07:54:14,667 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-04-09 07:54:14,667 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230409' 2023-04-09 07:54:14,667 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230409 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-04-09 07:54:14,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-04-09 07:54:14,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:14,668 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-04-09 07:54:14,670 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 896 msec 2023-04-09 07:54:14,673 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): f0431b4fc004fb6549afd7e745a9bb5c#family1#compaction#37 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:14,705 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=248 (was 249), OpenFileDescriptor=590 (was 596), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=298 (was 324), ProcessCount=169 (was 169), AvailableMemoryMB=1775 (was 1764) - AvailableMemoryMB LEAK? - 2023-04-09 07:54:14,717 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=248, OpenFileDescriptor=590, MaxFileDescriptor=60000, SystemLoadAverage=298, ProcessCount=169, AvailableMemoryMB=1775 2023-04-09 07:54:14,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:54:14,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:14,721 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:14,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-04-09 07:54:14,722 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:14,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:14,723 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:14,724 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d empty. 2023-04-09 07:54:14,724 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:14,724 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-04-09 07:54:14,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:15,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:15,133 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:15,135 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2c79a4f3105037e03dd30b14fe66503d, NAME => 'testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:15,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:15,492 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/.tmp/family1/53a8495b51aa437f9e3bebda7395a236 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/53a8495b51aa437f9e3bebda7395a236 2023-04-09 07:54:15,498 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f0431b4fc004fb6549afd7e745a9bb5c/family1 of f0431b4fc004fb6549afd7e745a9bb5c into 53a8495b51aa437f9e3bebda7395a236(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:15,498 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:54:15,498 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1681026854661; duration=0sec 2023-04-09 07:54:15,498 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing 2c79a4f3105037e03dd30b14fe66503d, disabling compactions & flushes 2023-04-09 07:54:15,542 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. after waiting 0 ms 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,542 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,542 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:15,544 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:15,545 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026855545"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026855545"}]},"ts":"1681026855545"} 2023-04-09 07:54:15,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_] to archive 2023-04-09 07:54:15,550 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:54:15,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:15,550 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:15,551 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026855550"}]},"ts":"1681026855550"} 2023-04-09 07:54:15,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/893b8fbfe56e4df58ade9c049dfbe9df 2023-04-09 07:54:15,552 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-04-09 07:54:15,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/55398a0504a2413b9ac4b68266ebc781 2023-04-09 07:54:15,554 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/family1/e3588d437e8848dbbde71f724d29b08c_SeqId_12_ 2023-04-09 07:54:15,556 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2c79a4f3105037e03dd30b14fe66503d, ASSIGN}] 2023-04-09 07:54:15,557 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2c79a4f3105037e03dd30b14fe66503d, ASSIGN 2023-04-09 07:54:15,558 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2c79a4f3105037e03dd30b14fe66503d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:15,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2c79a4f3105037e03dd30b14fe66503d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:15,709 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026855708"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026855708"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026855708"}]},"ts":"1681026855708"} 2023-04-09 07:54:15,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure 2c79a4f3105037e03dd30b14fe66503d, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:15,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:15,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2c79a4f3105037e03dd30b14fe66503d, NAME => 'testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:15,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,868 INFO [StoreOpener-2c79a4f3105037e03dd30b14fe66503d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,869 DEBUG [StoreOpener-2c79a4f3105037e03dd30b14fe66503d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:15,869 DEBUG [StoreOpener-2c79a4f3105037e03dd30b14fe66503d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:15,870 INFO [StoreOpener-2c79a4f3105037e03dd30b14fe66503d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c79a4f3105037e03dd30b14fe66503d columnFamilyName family1 2023-04-09 07:54:15,871 INFO [StoreOpener-2c79a4f3105037e03dd30b14fe66503d-1] regionserver.HStore(310): Store=2c79a4f3105037e03dd30b14fe66503d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:15,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:54:15,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:15,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2c79a4f3105037e03dd30b14fe66503d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69880590, jitterRate=0.04130193591117859}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:15,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:15,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d., pid=37, masterSystemTime=1681026855863 2023-04-09 07:54:15,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:15,881 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=2c79a4f3105037e03dd30b14fe66503d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:15,881 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1681026855881"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026855881"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026855881"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026855881"}]},"ts":"1681026855881"} 2023-04-09 07:54:15,884 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-04-09 07:54:15,885 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure 2c79a4f3105037e03dd30b14fe66503d, server=jenkins-hbase4.apache.org,34907,1681026816245 in 172 msec 2023-04-09 07:54:15,887 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-04-09 07:54:15,887 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=2c79a4f3105037e03dd30b14fe66503d, ASSIGN in 329 msec 2023-04-09 07:54:15,887 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:15,888 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026855887"}]},"ts":"1681026855887"} 2023-04-09 07:54:15,889 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-04-09 07:54:15,892 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:15,893 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1730 sec 2023-04-09 07:54:16,383 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:16,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-09 07:54:16,828 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-04-09 07:54:16,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:16,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:16,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:16,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:16,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:16,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:16,854 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:16,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:16,855 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:16,855 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,855 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,856 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:16,856 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:16,856 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:16,856 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:16,860 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:16,860 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:16,860 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:16,860 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:16,860 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:54:17,286 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f5f672146b184f75b925bc76578d57cb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5f672146b184f75b925bc76578d57cb 2023-04-09 07:54:17,690 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/3ccf2311643b48fba1e967aa7807b32e 2023-04-09 07:54:17,695 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/3ccf2311643b48fba1e967aa7807b32e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e 2023-04-09 07:54:17,699 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e, entries=1000, sequenceid=5, filesize=165.1 K 2023-04-09 07:54:17,700 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 840ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:17,700 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:17,701 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:17,701 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:17,701 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:17,701 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:17,701 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,701 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:17,701 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:17,703 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,703 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:17,703 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,703 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:17,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6106815[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:17,704 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:17,704 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,706 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,706 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:17,706 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:17,707 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:17,707 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,707 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:17,708 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:17,708 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,708 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:17,708 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:17,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:17,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:17,710 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:17,710 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:17,710 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:17,710 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:17,710 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,710 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:17,711 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,711 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:17,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,711 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:17,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:17,712 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:17,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:17,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:17,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:17,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,713 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:17,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:17,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:17,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:17,715 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:17,715 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:17,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:17,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:17,815 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:17,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:17,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:17,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:17,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,822 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:17,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:17,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:17,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:17,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,825 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,825 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:17,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:17,825 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:17,825 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:17,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:17,827 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:17,827 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:17,827 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:54:18,239 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511288f3592a61ab64da18124651cd65c3f05 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511288f3592a61ab64da18124651cd65c3f05 2023-04-09 07:54:18,644 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/da37f8d557c14a1f82bca6129a393295 2023-04-09 07:54:18,649 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/da37f8d557c14a1f82bca6129a393295 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295 2023-04-09 07:54:18,653 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:54:18,654 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 827ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:18,654 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:18,655 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:18,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:18,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:18,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:18,655 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:18,656 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,656 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:18,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:18,656 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,657 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:18,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:18,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1637f6f7[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:18,658 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:18,658 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,663 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,663 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,663 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:18,663 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:18,663 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:18,663 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:18,664 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:18,664 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,664 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,665 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:18,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:18,666 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:18,667 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:18,667 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:18,667 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,667 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:18,668 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,668 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:18,668 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,668 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:18,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:18,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:18,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:18,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,670 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,671 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,671 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:18,671 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:18,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:18,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:18,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:18,671 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:18,671 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,672 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:18,772 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:18,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:18,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:18,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:18,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,779 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:18,779 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:18,779 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:18,779 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:18,779 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,779 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,784 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,784 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:18,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:18,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:18,784 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:18,784 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:18,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:18,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,785 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,785 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:18,785 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:18,785 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:18,785 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:18,785 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:18,786 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:18,786 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:18,786 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:18,786 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:18,786 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:19,202 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120114e6a689ed644972aaad3600957eb3c3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120114e6a689ed644972aaad3600957eb3c3 2023-04-09 07:54:19,220 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/2135ec56f9354d1ba23d7849b013178d 2023-04-09 07:54:19,226 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/2135ec56f9354d1ba23d7849b013178d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d 2023-04-09 07:54:19,232 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d, entries=1, sequenceid=13, filesize=4.9 K 2023-04-09 07:54:19,233 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 447ms, sequenceid=13, compaction requested=true 2023-04-09 07:54:19,234 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:19,234 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:19,234 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:19,234 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:19,234 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,234 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:19,234 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:19,236 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:19,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:19,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:19,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:19,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:19,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:19,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:19,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6dc517b9[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:19,239 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:19,239 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,240 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,240 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:19,240 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:19,240 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:19,240 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,240 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:19,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:19,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:19,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:19,242 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:19,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:19,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:19,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:19,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:19,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:19,245 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:19,245 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:19,245 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:19,245 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,245 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:19,248 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,248 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:19,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,248 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:19,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:19,249 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:19,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:19,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:19,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:19,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,251 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:19,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:19,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,254 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:19,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:19,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:19,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:19,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:19,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:19,255 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:19,255 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:19,355 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:19,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:19,368 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:19,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:19,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,370 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:19,370 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:19,370 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:19,370 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:19,371 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,371 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,372 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:19,372 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:19,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:19,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,373 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:19,373 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:19,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:19,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:19,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:19,374 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:19,374 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:19,374 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:19,374 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:19,374 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:19,374 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:19,374 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:19,375 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:19,813 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051dbead3c99914541b934ce4ef5d09cc0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512051dbead3c99914541b934ce4ef5d09cc0 2023-04-09 07:54:20,223 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/1e9b9b49cc924eeb86632c1ff7cfdb59 2023-04-09 07:54:20,229 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/1e9b9b49cc924eeb86632c1ff7cfdb59 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59 2023-04-09 07:54:20,235 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59, entries=1, sequenceid=17, filesize=4.9 K 2023-04-09 07:54:20,236 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 861ms, sequenceid=17, compaction requested=true 2023-04-09 07:54:20,236 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:20,236 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:20,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:20,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:20,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:20,236 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:20,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,238 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,239 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,239 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:20,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24a447ad[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:20,240 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:20,241 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,243 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:20,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:20,243 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:20,243 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,243 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:20,244 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,244 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:20,245 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:20,245 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:20,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:20,247 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:20,247 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:20,247 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:20,247 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,247 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:20,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,250 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:20,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:20,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,251 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,253 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:20,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:20,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:20,255 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:20,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:20,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:20,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:20,255 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:20,256 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:20,256 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:20,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:20,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:20,356 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:20,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:20,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:20,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:20,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,364 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:20,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:20,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:20,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:20,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,365 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,366 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:20,366 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:20,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:20,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:20,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:20,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:20,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:20,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:20,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:20,369 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:20,369 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:20,369 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:20,369 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:20,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:20,798 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122822f18bdafc5b46f7ab686f2d01a0d65a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122822f18bdafc5b46f7ab686f2d01a0d65a 2023-04-09 07:54:20,813 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/933eeaa6679d461f9023d7d937945b36 2023-04-09 07:54:20,821 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/933eeaa6679d461f9023d7d937945b36 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36 2023-04-09 07:54:20,826 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36, entries=1, sequenceid=21, filesize=4.9 K 2023-04-09 07:54:20,827 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 458ms, sequenceid=21, compaction requested=true 2023-04-09 07:54:20,827 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:20,827 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:20,827 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:20,828 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:20,836 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,836 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,837 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,837 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:20,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2171d3ec[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:20,838 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:20,838 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,839 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:20,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:20,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:20,840 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,840 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:20,841 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:20,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:20,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:20,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:20,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:20,844 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:20,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:20,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:20,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,844 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:20,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:20,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:20,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:20,851 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:20,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:20,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,852 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:20,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,855 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:20,855 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:20,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:20,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:20,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:20,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:20,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:20,856 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:20,856 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:20,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:20,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:20,956 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:20,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:20,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:20,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:20,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,964 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:20,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:20,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:20,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:20,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,965 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:20,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:20,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:20,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:20,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:20,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:20,967 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:20,967 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:20,967 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:20,967 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:21,369 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:21,403 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231431bcf4338d540b3ad8a11d4b1e405ba to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231431bcf4338d540b3ad8a11d4b1e405ba 2023-04-09 07:54:21,823 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/a958f6d76b2546d5a25a9c9322c62726 2023-04-09 07:54:21,828 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/a958f6d76b2546d5a25a9c9322c62726 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726 2023-04-09 07:54:21,832 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726, entries=1, sequenceid=25, filesize=4.9 K 2023-04-09 07:54:21,833 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 866ms, sequenceid=25, compaction requested=true 2023-04-09 07:54:21,833 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:21,833 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:21,833 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:21,833 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:21,833 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,834 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:21,834 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:21,835 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:21,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:21,836 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,836 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:21,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:21,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8d8097b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:21,837 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:21,837 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,839 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,839 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:21,839 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:21,839 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:21,839 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,839 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:21,841 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:21,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:21,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:21,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:21,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:21,843 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:21,843 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:21,843 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:21,843 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,843 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:21,845 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,845 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,845 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:21,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:21,846 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:21,847 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:21,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:21,849 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:21,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:21,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:21,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:21,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:21,850 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:21,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:21,850 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:21,950 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:21,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:21,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:21,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:21,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,958 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:21,958 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:21,958 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:21,958 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:21,959 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,959 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,960 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:21,960 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:21,961 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:21,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,961 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:21,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:21,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:21,962 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:21,962 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:21,962 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:21,962 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:21,962 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:21,962 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:21,962 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:21,962 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:22,376 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e1380d59455a46b484e5d9bc24f8a10f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1380d59455a46b484e5d9bc24f8a10f 2023-04-09 07:54:22,782 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/8a1eae1a4a9c42419c17358addc90b0f 2023-04-09 07:54:22,788 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/8a1eae1a4a9c42419c17358addc90b0f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f 2023-04-09 07:54:22,792 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f, entries=1, sequenceid=29, filesize=4.9 K 2023-04-09 07:54:22,793 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 831ms, sequenceid=29, compaction requested=true 2023-04-09 07:54:22,794 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:22,794 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:22,794 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:22,795 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,795 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:22,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:22,796 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,796 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:22,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:22,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:22,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:22,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:22,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4de5bf06[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:22,798 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:22,798 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,799 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,799 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:22,799 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:22,799 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:22,799 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,799 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:22,801 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,801 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:22,801 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:22,801 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:22,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:22,803 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:22,803 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:22,803 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:22,803 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,803 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:22,805 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,805 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,806 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:22,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:22,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:22,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:22,806 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:22,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,808 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:22,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,810 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,810 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,810 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,810 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:22,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:22,810 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:22,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:22,810 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:22,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:22,811 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:22,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:22,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:22,911 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:22,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:22,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:22,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:22,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,921 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:22,921 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:22,921 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:22,921 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:22,922 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,922 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:22,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:22,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:22,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:22,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:22,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:22,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,927 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:22,927 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:22,927 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:22,927 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:22,927 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:22,927 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:23,353 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036f79a573556f49039378006250f19be1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036f79a573556f49039378006250f19be1 2023-04-09 07:54:23,360 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/ac0e67a1c496436d8152bcfcb2a67eae 2023-04-09 07:54:23,366 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/ac0e67a1c496436d8152bcfcb2a67eae as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae 2023-04-09 07:54:23,370 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae, entries=1, sequenceid=33, filesize=4.9 K 2023-04-09 07:54:23,371 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 444ms, sequenceid=33, compaction requested=true 2023-04-09 07:54:23,371 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:23,371 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:23,373 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:23,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:23,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:23,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:23,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:23,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:23,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:23,375 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b7a1038[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:23,375 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:23,375 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,376 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,377 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:23,377 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:23,377 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:23,377 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,377 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:23,378 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:23,378 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,378 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:23,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:23,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:23,378 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:23,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:23,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:23,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:23,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:23,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:23,381 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:23,381 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:23,381 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:23,381 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,381 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:23,383 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:23,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:23,383 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,384 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:23,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:23,384 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,385 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:23,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:23,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:23,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:23,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:23,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:23,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:23,387 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:23,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:23,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:23,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,388 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,488 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:23,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:23,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:23,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:23,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,496 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:23,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:23,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:23,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:23,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,496 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,498 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:23,498 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:23,498 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:23,498 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:23,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,499 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:23,499 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:23,499 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:23,500 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:23,500 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:23,500 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:23,500 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:23,500 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:23,915 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509070d76189d95e14c1bb92e96cf5a0cbc2e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201509070d76189d95e14c1bb92e96cf5a0cbc2e 2023-04-09 07:54:24,327 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/5cf94c69ecf141d18766c45967b9eb4e 2023-04-09 07:54:24,333 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/5cf94c69ecf141d18766c45967b9eb4e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e 2023-04-09 07:54:24,338 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e, entries=1, sequenceid=37, filesize=4.9 K 2023-04-09 07:54:24,339 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 839ms, sequenceid=37, compaction requested=true 2023-04-09 07:54:24,339 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:24,339 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:24,340 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:24,340 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:24,340 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,340 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:24,340 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:24,341 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:24,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:24,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64bb5408[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:24,343 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:24,343 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,345 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:24,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:24,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:24,346 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,346 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:24,347 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,347 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:24,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:24,350 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:24,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:24,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:24,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,350 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:24,352 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,352 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,352 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:24,352 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:24,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,354 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:24,356 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:24,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:24,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:24,356 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:24,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:24,356 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:24,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:24,457 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:24,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:24,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:24,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:24,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,466 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:24,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:24,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-04-09 07:54:24,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-04-09 07:54:24,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,469 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:24,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:24,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:24,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,469 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:24,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:24,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,470 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,471 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-04-09 07:54:24,471 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-04-09 07:54:24,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:24,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-04-09 07:54:24,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:24,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:24,471 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:24,471 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. started... 2023-04-09 07:54:24,471 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2c79a4f3105037e03dd30b14fe66503d 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:24,485 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112030df0ba2a68a4374b13edce5c56138e6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112030df0ba2a68a4374b13edce5c56138e6 2023-04-09 07:54:24,894 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/7f20b9034f8f4576ae88f43f4524798d 2023-04-09 07:54:24,900 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/7f20b9034f8f4576ae88f43f4524798d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d 2023-04-09 07:54:24,905 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d, entries=1, sequenceid=41, filesize=4.9 K 2023-04-09 07:54:24,906 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 2c79a4f3105037e03dd30b14fe66503d in 435ms, sequenceid=41, compaction requested=true 2023-04-09 07:54:24,906 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:24,906 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:24,906 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:24,906 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:24,906 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,906 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-04-09 07:54:24,906 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-04-09 07:54:24,908 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,908 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,908 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,908 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-04-09 07:54:24,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c600a35[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:24,910 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-04-09 07:54:24,910 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,911 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,911 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-04-09 07:54:24,911 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-04-09 07:54:24,911 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:24,911 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,911 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:24,913 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,913 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-04-09 07:54:24,913 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:24,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,914 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-04-09 07:54:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:24,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:54:24,916 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-04-09 07:54:24,916 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:24,916 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:24,916 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,917 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:24,918 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,918 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:24,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,918 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:24,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:24,919 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:24,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,920 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:24,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:24,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:24,925 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:24,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,926 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:24,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-04-09 07:54:24,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:24,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-04-09 07:54:24,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:24,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:24,926 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:24,926 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:25,026 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:25,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:25,028 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-04-09 07:54:25,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:25,031 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:25,032 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:25,032 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027465032 2023-04-09 07:54:25,033 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:25,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-04-09 07:54:25,035 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,035 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x4ee22ef6 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:25,040 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d72562d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:25,040 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,093 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034295a33224e948c1b6836fdd681af17e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601034295a33224e948c1b6836fdd681af17e 2023-04-09 07:54:25,094 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,112 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:25,114 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43404, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:25,121 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:25,122 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52672, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:25,134 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/da23f748534549e2a8ff61c81772cfd6 first=Optional[r3] last=Optional[r6] 2023-04-09 07:54:25,135 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/da23f748534549e2a8ff61c81772cfd6}] 2023-04-09 07:54:25,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/da23f748534549e2a8ff61c81772cfd6 for inclusion in 2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:25,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-09 07:54:25,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:25,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:25,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/da23f748534549e2a8ff61c81772cfd6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jek7groqdjjp3f0j5fblgeist4cknklrd0mp6am4rr5cell46tq96em6rrqvl4lv/family1/da23f748534549e2a8ff61c81772cfd6 2023-04-09 07:54:25,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jek7groqdjjp3f0j5fblgeist4cknklrd0mp6am4rr5cell46tq96em6rrqvl4lv/family1/da23f748534549e2a8ff61c81772cfd6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_ 2023-04-09 07:54:25,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jek7groqdjjp3f0j5fblgeist4cknklrd0mp6am4rr5cell46tq96em6rrqvl4lv/family1/da23f748534549e2a8ff61c81772cfd6 into 2c79a4f3105037e03dd30b14fe66503d/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_ - updating store file list. 2023-04-09 07:54:25,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_ into 2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:25,151 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jek7groqdjjp3f0j5fblgeist4cknklrd0mp6am4rr5cell46tq96em6rrqvl4lv/family1/da23f748534549e2a8ff61c81772cfd6 into 2c79a4f3105037e03dd30b14fe66503d/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_) 2023-04-09 07:54:25,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__jek7groqdjjp3f0j5fblgeist4cknklrd0mp6am4rr5cell46tq96em6rrqvl4lv/family1/da23f748534549e2a8ff61c81772cfd6 2023-04-09 07:54:25,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:54:25,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:54:25,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 2c79a4f3105037e03dd30b14fe66503d/family1 is initiating minor compaction 2023-04-09 07:54:25,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@434d26c3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:25,155 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c79a4f3105037e03dd30b14fe66503d/family1 in testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:25,156 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp, totalSize=50.0 K 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting da37f8d557c14a1f82bca6129a393295, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 2135ec56f9354d1ba23d7849b013178d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 1e9b9b49cc924eeb86632c1ff7cfdb59, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 933eeaa6679d461f9023d7d937945b36, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting a958f6d76b2546d5a25a9c9322c62726, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 8a1eae1a4a9c42419c17358addc90b0f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting ac0e67a1c496436d8152bcfcb2a67eae, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5cf94c69ecf141d18766c45967b9eb4e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 7f20b9034f8f4576ae88f43f4524798d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:54:25,156 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:54:25,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,158 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:25,160 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122822f18bdafc5b46f7ab686f2d01a0d65a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122822f18bdafc5b46f7ab686f2d01a0d65a 2023-04-09 07:54:25,161 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231431bcf4338d540b3ad8a11d4b1e405ba to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231431bcf4338d540b3ad8a11d4b1e405ba 2023-04-09 07:54:25,162 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1380d59455a46b484e5d9bc24f8a10f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101e1380d59455a46b484e5d9bc24f8a10f 2023-04-09 07:54:25,164 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036f79a573556f49039378006250f19be1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601036f79a573556f49039378006250f19be1 2023-04-09 07:54:25,164 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-04-09 07:54:25,164 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,165 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,212 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2c79a4f3105037e03dd30b14fe66503d#family1#compaction#48 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:25,289 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/0a5111bd15054200a74a6973b4803b65 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0a5111bd15054200a74a6973b4803b65 2023-04-09 07:54:25,297 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120530bf745ee1ed4b8592388e82a601c2b5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120530bf745ee1ed4b8592388e82a601c2b5 2023-04-09 07:54:25,299 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 2c79a4f3105037e03dd30b14fe66503d/family1 of 2c79a4f3105037e03dd30b14fe66503d into 0a5111bd15054200a74a6973b4803b65(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:25,299 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:25,299 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1681026865155; duration=0sec 2023-04-09 07:54:25,300 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:25,300 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,374 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_] to archive 2023-04-09 07:54:25,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:25,378 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/da37f8d557c14a1f82bca6129a393295 2023-04-09 07:54:25,379 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/21710845519d4896a3f6cca7961ada99 first=Optional[r00] last=Optional[r2] 2023-04-09 07:54:25,380 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/21710845519d4896a3f6cca7961ada99}] 2023-04-09 07:54:25,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/2135ec56f9354d1ba23d7849b013178d 2023-04-09 07:54:25,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/1e9b9b49cc924eeb86632c1ff7cfdb59 2023-04-09 07:54:25,383 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/21710845519d4896a3f6cca7961ada99 for inclusion in 2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:25,384 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/933eeaa6679d461f9023d7d937945b36 2023-04-09 07:54:25,385 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/a958f6d76b2546d5a25a9c9322c62726 2023-04-09 07:54:25,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-09 07:54:25,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:25,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:25,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/21710845519d4896a3f6cca7961ada99 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r8tqptgfe19ubnd26hnc0bgb6lvo6ink1v4df6cc6agqcvf4j8pm7slvd2ng7p6t/family1/21710845519d4896a3f6cca7961ada99 2023-04-09 07:54:25,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/8a1eae1a4a9c42419c17358addc90b0f 2023-04-09 07:54:25,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r8tqptgfe19ubnd26hnc0bgb6lvo6ink1v4df6cc6agqcvf4j8pm7slvd2ng7p6t/family1/21710845519d4896a3f6cca7961ada99 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_ 2023-04-09 07:54:25,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r8tqptgfe19ubnd26hnc0bgb6lvo6ink1v4df6cc6agqcvf4j8pm7slvd2ng7p6t/family1/21710845519d4896a3f6cca7961ada99 into 2c79a4f3105037e03dd30b14fe66503d/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_ - updating store file list. 2023-04-09 07:54:25,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/ac0e67a1c496436d8152bcfcb2a67eae 2023-04-09 07:54:25,393 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/5cf94c69ecf141d18766c45967b9eb4e 2023-04-09 07:54:25,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/7f20b9034f8f4576ae88f43f4524798d 2023-04-09 07:54:25,396 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/54bed73cbdbb4d8aaff8c18640e1014a_SeqId_44_ 2023-04-09 07:54:25,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_ into 2c79a4f3105037e03dd30b14fe66503d/family1 2023-04-09 07:54:25,398 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r8tqptgfe19ubnd26hnc0bgb6lvo6ink1v4df6cc6agqcvf4j8pm7slvd2ng7p6t/family1/21710845519d4896a3f6cca7961ada99 into 2c79a4f3105037e03dd30b14fe66503d/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_) 2023-04-09 07:54:25,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyPolicy__r8tqptgfe19ubnd26hnc0bgb6lvo6ink1v4df6cc6agqcvf4j8pm7slvd2ng7p6t/family1/21710845519d4896a3f6cca7961ada99 2023-04-09 07:54:25,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:54:25,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:54:25,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 2c79a4f3105037e03dd30b14fe66503d/family1 is initiating minor compaction (all files) 2023-04-09 07:54:25,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64812fe2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:25,400 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2c79a4f3105037e03dd30b14fe66503d/family1 in testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:54:25,400 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0a5111bd15054200a74a6973b4803b65, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp, totalSize=337.0 K 2023-04-09 07:54:25,402 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 3ccf2311643b48fba1e967aa7807b32e, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:54:25,402 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 0a5111bd15054200a74a6973b4803b65, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:54:25,403 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 0e69668084b84de4bc992a3a38b9314f_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840049 2023-04-09 07:54:25,403 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,404 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:25,406 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5f672146b184f75b925bc76578d57cb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130f5f672146b184f75b925bc76578d57cb 2023-04-09 07:54:25,410 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120114e6a689ed644972aaad3600957eb3c3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120114e6a689ed644972aaad3600957eb3c3 2023-04-09 07:54:25,411 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512051dbead3c99914541b934ce4ef5d09cc0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512051dbead3c99914541b934ce4ef5d09cc0 2023-04-09 07:54:25,411 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-04-09 07:54:25,411 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-09 07:54:25,412 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-04-09 07:54:25,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-04-09 07:54:25,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:25,413 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:54:25,416 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 384 msec 2023-04-09 07:54:25,431 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2c79a4f3105037e03dd30b14fe66503d#family1#compaction#49 average throughput is 8.14 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:25,490 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/.tmp/family1/e7cd2001c6d241bb96ef64fec24fa5a4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/e7cd2001c6d241bb96ef64fec24fa5a4 2023-04-09 07:54:25,503 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 2c79a4f3105037e03dd30b14fe66503d/family1 of 2c79a4f3105037e03dd30b14fe66503d into e7cd2001c6d241bb96ef64fec24fa5a4(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:25,503 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:54:25,504 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1681026865400; duration=0sec 2023-04-09 07:54:25,504 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:25,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0a5111bd15054200a74a6973b4803b65, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_] to archive 2023-04-09 07:54:25,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:25,559 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/3ccf2311643b48fba1e967aa7807b32e 2023-04-09 07:54:25,562 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0a5111bd15054200a74a6973b4803b65 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0a5111bd15054200a74a6973b4803b65 2023-04-09 07:54:25,564 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/family1/0e69668084b84de4bc992a3a38b9314f_SeqId_47_ 2023-04-09 07:54:25,574 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=259 (was 248) - Thread LEAK? -, OpenFileDescriptor=626 (was 590) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=336 (was 298) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1730 (was 1775) 2023-04-09 07:54:25,590 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=259, OpenFileDescriptor=628, MaxFileDescriptor=60000, SystemLoadAverage=336, ProcessCount=169, AvailableMemoryMB=1729 2023-04-09 07:54:25,594 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:54:25,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:25,602 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:25,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-04-09 07:54:25,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-09 07:54:25,605 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:25,613 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:25,613 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 empty. 2023-04-09 07:54:25,614 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:25,614 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-04-09 07:54:25,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-09 07:54:25,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-09 07:54:26,039 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:26,040 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 33695bd4d17ece9a6c2338030063f345, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:26,051 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:26,051 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 33695bd4d17ece9a6c2338030063f345, disabling compactions & flushes 2023-04-09 07:54:26,052 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,052 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,052 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. after waiting 0 ms 2023-04-09 07:54:26,052 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,052 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,052 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:26,054 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:26,055 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026866055"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026866055"}]},"ts":"1681026866055"} 2023-04-09 07:54:26,057 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:54:26,058 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:26,058 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026866058"}]},"ts":"1681026866058"} 2023-04-09 07:54:26,059 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-04-09 07:54:26,062 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, ASSIGN}] 2023-04-09 07:54:26,064 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, ASSIGN 2023-04-09 07:54:26,065 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:26,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-09 07:54:26,216 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:26,216 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026866216"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026866216"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026866216"}]},"ts":"1681026866216"} 2023-04-09 07:54:26,218 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:26,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 33695bd4d17ece9a6c2338030063f345, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:26,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:26,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,377 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,379 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:26,379 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:26,383 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33695bd4d17ece9a6c2338030063f345 columnFamilyName family1 2023-04-09 07:54:26,385 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(310): Store=33695bd4d17ece9a6c2338030063f345/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:26,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:26,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:26,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 33695bd4d17ece9a6c2338030063f345; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73113927, jitterRate=0.08948241174221039}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:26,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:26,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., pid=41, masterSystemTime=1681026866370 2023-04-09 07:54:26,397 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,398 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:26,398 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026866397"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026866397"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026866397"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026866397"}]},"ts":"1681026866397"} 2023-04-09 07:54:26,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-04-09 07:54:26,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245 in 181 msec 2023-04-09 07:54:26,403 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-04-09 07:54:26,403 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, ASSIGN in 339 msec 2023-04-09 07:54:26,404 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:26,404 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026866404"}]},"ts":"1681026866404"} 2023-04-09 07:54:26,406 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-04-09 07:54:26,410 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:26,412 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 816 msec 2023-04-09 07:54:26,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-04-09 07:54:26,709 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-04-09 07:54:26,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:26,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,742 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:26,742 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:26,742 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:26,743 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:26,743 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,743 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:26,744 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:26,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:26,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:26,745 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:26,745 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:26,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:26,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:26,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:26,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:26,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:26,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:26,748 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:26,748 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:26,748 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:26,749 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:54:27,200 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130142171c80aae40e0b5828e7a4d193653 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130142171c80aae40e0b5828e7a4d193653 2023-04-09 07:54:27,605 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/be99707cc6804cb999c2be5f95260dce 2023-04-09 07:54:27,613 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/be99707cc6804cb999c2be5f95260dce as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce 2023-04-09 07:54:27,621 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce, entries=1000, sequenceid=5, filesize=191.4 K 2023-04-09 07:54:27,623 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 874ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:27,623 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:27,624 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:27,624 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:27,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:27,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:27,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:27,624 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:27,626 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,626 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:27,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:27,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c9d8d30[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:27,627 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:27,627 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,631 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:27,632 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:27,632 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,632 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:27,640 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:27,640 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:27,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:27,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:27,643 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:27,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:27,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:27,643 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,643 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:27,645 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,645 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,645 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:27,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:27,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:27,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,648 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,649 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:27,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:27,652 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:27,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:27,652 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:27,652 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:27,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:27,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:27,753 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:27,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:27,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:27,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:27,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,792 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:27,792 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:27,792 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:27,792 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:27,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,795 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:27,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:27,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:27,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,796 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,796 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:27,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,796 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,797 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:27,797 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:27,805 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:27,805 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:27,805 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:27,805 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:27,805 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:27,808 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:27,808 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:27,808 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:27,808 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:27,808 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:54:27,858 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128316a6aed8c284f78ac425084c9823fdb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128316a6aed8c284f78ac425084c9823fdb 2023-04-09 07:54:28,265 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/0635494ceafe44a8884e0fe6e55feeb4 2023-04-09 07:54:28,270 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/0635494ceafe44a8884e0fe6e55feeb4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4 2023-04-09 07:54:28,281 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:54:28,282 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 474ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:28,282 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:28,282 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:28,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,290 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:28,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:28,291 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,291 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@577daf4b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:28,293 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:28,293 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,294 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:28,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:28,294 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:28,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,295 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:28,296 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,296 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:28,296 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:28,296 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:28,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:28,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:28,299 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:28,299 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:28,299 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:28,299 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,299 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:28,300 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,300 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:28,300 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:28,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:28,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:28,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:28,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:28,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:28,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,303 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:28,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:28,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:28,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:28,310 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,310 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:28,311 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:28,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:28,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:28,411 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:28,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:28,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:28,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:28,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,419 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:28,419 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:28,420 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:28,420 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:28,420 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,420 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,422 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:28,422 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:28,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:28,422 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:28,422 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:28,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:28,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:28,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:28,423 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:28,423 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:28,423 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:28,424 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:28,424 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:28,424 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:28,424 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:28,424 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:28,845 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512012c2f9597c8394cd58fb3b186b0b6b3a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512012c2f9597c8394cd58fb3b186b0b6b3a7 2023-04-09 07:54:29,252 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/332a3266a32c4e65b71791247d2c2df8 2023-04-09 07:54:29,260 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/332a3266a32c4e65b71791247d2c2df8 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8 2023-04-09 07:54:29,266 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8, entries=1, sequenceid=13, filesize=5.0 K 2023-04-09 07:54:29,267 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 843ms, sequenceid=13, compaction requested=true 2023-04-09 07:54:29,268 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:29,268 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:29,268 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:29,268 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:29,268 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,268 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:29,268 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:29,272 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,272 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,273 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,273 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:29,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6958aab8[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:29,275 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:29,275 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,277 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:29,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:29,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:29,278 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,278 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:29,279 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,279 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:29,279 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:29,279 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:29,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:29,283 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:29,283 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:29,283 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:29,283 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,283 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:29,285 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,286 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,287 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,289 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,289 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:29,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,290 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,290 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:29,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:29,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:29,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:29,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:29,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:29,290 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,291 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,291 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:29,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:29,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:29,291 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:29,391 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:29,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:29,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:29,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:29,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,398 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:29,398 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:29,398 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:29,398 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:29,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,399 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,400 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:29,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:29,400 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,400 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,400 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:29,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:29,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:29,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:29,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:29,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:29,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:29,402 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:29,402 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:29,402 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:29,402 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:29,424 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f4b1256d764e4c90b2baba4a078c3ada to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4b1256d764e4c90b2baba4a078c3ada 2023-04-09 07:54:29,831 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/f854bb948a0b48d5830e7d8fcb0195cd 2023-04-09 07:54:29,837 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/f854bb948a0b48d5830e7d8fcb0195cd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd 2023-04-09 07:54:29,841 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd, entries=1, sequenceid=17, filesize=5.0 K 2023-04-09 07:54:29,842 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 440ms, sequenceid=17, compaction requested=true 2023-04-09 07:54:29,842 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:29,842 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:29,844 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:29,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45c53404[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:29,845 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:29,846 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,847 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:29,847 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,847 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:29,847 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:29,849 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:29,849 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:29,849 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,849 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:29,852 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:29,852 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:29,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:29,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:29,853 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,853 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:29,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,855 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:29,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,856 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:29,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:29,856 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,858 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:29,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,860 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:29,860 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:29,860 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:29,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:29,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:29,860 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:29,860 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:29,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:29,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:29,861 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:29,961 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:29,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:29,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:29,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:29,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,968 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:29,968 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:29,968 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:29,969 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:29,969 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,969 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,970 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:29,970 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:29,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:29,971 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:29,971 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,972 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:29,972 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:29,972 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:29,972 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:29,972 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:30,393 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c58ebad68bf74cf48f65699a04964fab to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c58ebad68bf74cf48f65699a04964fab 2023-04-09 07:54:30,800 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/03cd4b2e80a94648bdd256519b80910f 2023-04-09 07:54:30,806 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/03cd4b2e80a94648bdd256519b80910f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f 2023-04-09 07:54:30,810 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f, entries=1, sequenceid=21, filesize=5.0 K 2023-04-09 07:54:30,811 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 839ms, sequenceid=21, compaction requested=true 2023-04-09 07:54:30,812 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:30,812 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:30,814 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,814 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:30,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,815 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:30,816 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:30,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6591d581[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:30,817 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:30,817 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,821 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,822 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:30,822 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:30,822 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:30,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,822 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:30,823 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,823 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:30,823 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:30,823 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:30,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:30,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:30,826 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:30,826 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:30,826 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:30,826 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,826 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:30,828 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,828 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:30,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:30,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,828 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:30,829 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,830 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:30,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:30,833 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:30,833 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:30,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:30,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:30,834 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:30,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:30,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:30,834 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:30,834 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:30,934 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:30,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:30,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:30,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:30,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,942 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:30,942 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:30,942 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:30,942 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:30,943 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,943 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,944 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:30,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:30,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:30,944 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:30,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:30,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:30,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,948 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:30,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:30,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:30,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:30,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:30,951 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:30,951 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:30,951 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:30,951 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:30,951 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:31,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512310b46ca73a803494f9eaba0561f7dcbe3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310b46ca73a803494f9eaba0561f7dcbe3 2023-04-09 07:54:31,374 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/075078c5f5ed46ba99acb19eab365bbb 2023-04-09 07:54:31,379 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/075078c5f5ed46ba99acb19eab365bbb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb 2023-04-09 07:54:31,384 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb, entries=1, sequenceid=25, filesize=5.0 K 2023-04-09 07:54:31,385 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 434ms, sequenceid=25, compaction requested=true 2023-04-09 07:54:31,385 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:31,385 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:31,385 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:31,385 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:31,385 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,385 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:31,385 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:31,387 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,387 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,387 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,387 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:31,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:31,389 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:31,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f2c298f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:31,389 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,391 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,391 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:31,391 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:31,391 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:31,391 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,391 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:31,392 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:31,392 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,392 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:31,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,392 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:31,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:31,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:31,395 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:31,395 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:31,395 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:31,395 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,395 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:31,397 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,397 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,397 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:31,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,397 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:31,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:31,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,398 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:31,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:31,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:31,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:31,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:31,401 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:31,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:31,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:31,502 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:31,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:31,507 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:31,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:31,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,510 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:31,510 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:31,510 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:31,510 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:31,510 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,510 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,512 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,512 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:31,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:31,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,513 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,514 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:31,514 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,514 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:31,514 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:31,514 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:31,514 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:31,514 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:31,514 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:31,514 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:31,514 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:31,527 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018ad687862f094645b6186ecabc13a0be to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601018ad687862f094645b6186ecabc13a0be 2023-04-09 07:54:31,881 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:31,934 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/5849212bf64f43dca380713dbadaecce 2023-04-09 07:54:31,939 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/5849212bf64f43dca380713dbadaecce as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce 2023-04-09 07:54:31,944 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce, entries=1, sequenceid=29, filesize=5.0 K 2023-04-09 07:54:31,945 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 431ms, sequenceid=29, compaction requested=true 2023-04-09 07:54:31,945 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:31,945 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:31,945 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:31,945 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:31,945 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,945 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:31,945 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:31,947 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,947 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:31,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@738d55b8[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:31,949 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:31,949 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,950 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:31,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:31,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:31,951 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,951 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:31,952 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:31,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:31,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:31,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:31,955 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:31,955 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:31,955 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:31,955 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,955 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:31,957 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,957 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:31,958 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:31,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,958 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:31,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:31,960 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:31,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:31,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:31,967 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:31,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:31,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:31,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:31,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:31,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:31,967 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:32,067 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:32,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:32,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:32,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:32,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,074 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:32,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:32,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:32,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:32,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,076 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,076 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:32,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:32,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:32,077 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,077 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:32,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:32,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:32,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:32,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:32,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:32,079 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:32,079 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:32,079 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:32,079 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:32,079 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:32,493 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d3b91977803e45f4a29ed3aa55e5adb2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d3b91977803e45f4a29ed3aa55e5adb2 2023-04-09 07:54:32,499 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/57d870393a5549a59cf377345ae62f3a 2023-04-09 07:54:32,503 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/57d870393a5549a59cf377345ae62f3a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a 2023-04-09 07:54:32,533 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:32,533 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ee22ef6 to 127.0.0.1:57867 2023-04-09 07:54:32,533 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:32,533 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a, entries=1, sequenceid=33, filesize=5.0 K 2023-04-09 07:54:32,533 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:32,533 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x35fecb8d to 127.0.0.1:57867 2023-04-09 07:54:32,533 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:32,534 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:54:32,534 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x22b81f9d to 127.0.0.1:57867 2023-04-09 07:54:32,534 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:54:32,537 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 458ms, sequenceid=33, compaction requested=true 2023-04-09 07:54:32,537 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:32,537 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:32,537 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:32,537 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:32,537 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,538 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:32,538 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:32,539 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,540 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,540 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:32,540 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:32,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:32,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:32,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:32,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:32,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:32,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f042ad9[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:32,544 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:32,544 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,545 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:32,545 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:32,546 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,546 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:32,547 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,547 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:32,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:32,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:32,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:32,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:32,550 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:32,550 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:32,550 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:32,550 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,550 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:32,555 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,555 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:32,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,555 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:32,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,557 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:32,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,558 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:32,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,561 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:32,561 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:32,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:32,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:32,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:32,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:32,562 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,562 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,562 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:32,562 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:32,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:32,562 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,562 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:32,563 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,563 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,563 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,563 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:32,563 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:32,663 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:32,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:32,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:32,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:32,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,671 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:32,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:32,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:32,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:32,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,671 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,673 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:32,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:32,673 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,673 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:32,673 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:32,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:32,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,674 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,674 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:32,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:32,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:32,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:32,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:32,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:32,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:32,675 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:32,675 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:32,675 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:33,091 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509075f6104f2dce74ca28b917d0a2592334d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509075f6104f2dce74ca28b917d0a2592334d 2023-04-09 07:54:33,126 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/2062a3ae024a433b90ddf0258f49a415 2023-04-09 07:54:33,131 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/2062a3ae024a433b90ddf0258f49a415 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415 2023-04-09 07:54:33,137 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415, entries=1, sequenceid=37, filesize=5.0 K 2023-04-09 07:54:33,138 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 463ms, sequenceid=37, compaction requested=true 2023-04-09 07:54:33,138 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:33,138 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:33,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:33,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:33,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:33,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:33,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,140 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:33,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:33,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74d2d965[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:33,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:33,142 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,144 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,144 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:33,144 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:33,144 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:33,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,145 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:33,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:33,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:33,146 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:33,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:33,148 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:33,148 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:33,148 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:33,148 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,148 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:33,150 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,150 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,150 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:33,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:33,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,150 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,152 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:33,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:33,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:33,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:33,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,162 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:33,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:33,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:33,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:33,163 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:33,263 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:33,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:33,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:33,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:33,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,271 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:33,271 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:33,271 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-04-09 07:54:33,271 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-04-09 07:54:33,271 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,271 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,273 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:33,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:33,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:33,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:33,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:33,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-04-09 07:54:33,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,274 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-04-09 07:54:33,275 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:33,275 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-04-09 07:54:33,275 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:33,275 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:33,275 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:33,275 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. started... 2023-04-09 07:54:33,275 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 33695bd4d17ece9a6c2338030063f345 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:33,291 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511205fed0b085e5e4ed1acaca663abb01d2e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511205fed0b085e5e4ed1acaca663abb01d2e 2023-04-09 07:54:33,698 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/a91a0e2af2614a50a4d2e05324013ed7 2023-04-09 07:54:33,703 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/a91a0e2af2614a50a4d2e05324013ed7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7 2023-04-09 07:54:33,708 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7, entries=1, sequenceid=41, filesize=5.0 K 2023-04-09 07:54:33,709 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 33695bd4d17ece9a6c2338030063f345 in 434ms, sequenceid=41, compaction requested=true 2023-04-09 07:54:33,709 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:33,709 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:33,709 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:33,709 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:33,709 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,709 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-04-09 07:54:33,709 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-04-09 07:54:33,711 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,711 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,712 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,712 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-04-09 07:54:33,713 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a0ce621[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:33,713 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-04-09 07:54:33,713 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,715 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,715 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-04-09 07:54:33,715 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-04-09 07:54:33,715 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:33,715 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,715 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:33,716 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,716 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-04-09 07:54:33,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:33,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-04-09 07:54:33,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:33,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-04-09 07:54:33,719 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-04-09 07:54:33,719 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:33,719 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:33,719 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,719 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:33,723 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,723 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,723 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:33,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:33,724 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,725 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:33,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:33,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:33,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:33,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:33,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-04-09 07:54:33,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-04-09 07:54:33,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:33,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:33,728 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:33,829 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:33,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:33,831 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:33,833 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:33,834 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:33,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:33,834 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027473834 2023-04-09 07:54:33,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:33,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-09 07:54:33,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:33,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x34df3cb3 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:33,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b13d73b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:33,842 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,697 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103952e1a0f42e84ba58ee79c1672ab76f7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103952e1a0f42e84ba58ee79c1672ab76f7 2023-04-09 07:54:34,699 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:34,712 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:34,713 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54538, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:34,720 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:34,722 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48910, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:34,736 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4892005a33c1423bbd37bc4d91be25a9 first=Optional[r3] last=Optional[r6] 2023-04-09 07:54:34,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4892005a33c1423bbd37bc4d91be25a9}] 2023-04-09 07:54:34,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4892005a33c1423bbd37bc4d91be25a9 for inclusion in 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:34,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-09 07:54:34,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:34,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:34,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4892005a33c1423bbd37bc4d91be25a9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mjse0g8s8kshrpba5v1600upkbcrmc1332iap5g4mufou12e7jib8kf5lbcf9bfm/family1/4892005a33c1423bbd37bc4d91be25a9 2023-04-09 07:54:34,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mjse0g8s8kshrpba5v1600upkbcrmc1332iap5g4mufou12e7jib8kf5lbcf9bfm/family1/4892005a33c1423bbd37bc4d91be25a9 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_ 2023-04-09 07:54:34,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mjse0g8s8kshrpba5v1600upkbcrmc1332iap5g4mufou12e7jib8kf5lbcf9bfm/family1/4892005a33c1423bbd37bc4d91be25a9 into 33695bd4d17ece9a6c2338030063f345/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_ - updating store file list. 2023-04-09 07:54:34,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_ into 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:34,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mjse0g8s8kshrpba5v1600upkbcrmc1332iap5g4mufou12e7jib8kf5lbcf9bfm/family1/4892005a33c1423bbd37bc4d91be25a9 into 33695bd4d17ece9a6c2338030063f345/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_) 2023-04-09 07:54:34,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mjse0g8s8kshrpba5v1600upkbcrmc1332iap5g4mufou12e7jib8kf5lbcf9bfm/family1/4892005a33c1423bbd37bc4d91be25a9 2023-04-09 07:54:34,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:54:34,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:54:34,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 33695bd4d17ece9a6c2338030063f345/family1 is initiating minor compaction 2023-04-09 07:54:34,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@64f5301e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:34,762 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 33695bd4d17ece9a6c2338030063f345/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:34,763 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp, totalSize=50.4 K 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 0635494ceafe44a8884e0fe6e55feeb4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 332a3266a32c4e65b71791247d2c2df8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting f854bb948a0b48d5830e7d8fcb0195cd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 03cd4b2e80a94648bdd256519b80910f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 075078c5f5ed46ba99acb19eab365bbb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5849212bf64f43dca380713dbadaecce, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 57d870393a5549a59cf377345ae62f3a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 2062a3ae024a433b90ddf0258f49a415, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting a91a0e2af2614a50a4d2e05324013ed7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:54:34,763 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting fc19282fb0d944518927b46895e4da1f_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:54:34,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:34,767 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:34,770 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c58ebad68bf74cf48f65699a04964fab to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228c58ebad68bf74cf48f65699a04964fab 2023-04-09 07:54:34,771 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310b46ca73a803494f9eaba0561f7dcbe3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512310b46ca73a803494f9eaba0561f7dcbe3 2023-04-09 07:54:34,773 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601018ad687862f094645b6186ecabc13a0be to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601018ad687862f094645b6186ecabc13a0be 2023-04-09 07:54:34,776 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d3b91977803e45f4a29ed3aa55e5adb2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d3b91977803e45f4a29ed3aa55e5adb2 2023-04-09 07:54:34,776 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-04-09 07:54:34,776 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:34,819 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 33695bd4d17ece9a6c2338030063f345#family1#compaction#60 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:34,880 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/c33c2d9133044751a0df596a463495ab as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/c33c2d9133044751a0df596a463495ab 2023-04-09 07:54:34,885 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 33695bd4d17ece9a6c2338030063f345/family1 of 33695bd4d17ece9a6c2338030063f345 into c33c2d9133044751a0df596a463495ab(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:34,885 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:34,886 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1681026874762; duration=0sec 2023-04-09 07:54:34,886 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:34,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_] to archive 2023-04-09 07:54:34,965 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:34,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/0635494ceafe44a8884e0fe6e55feeb4 2023-04-09 07:54:34,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/332a3266a32c4e65b71791247d2c2df8 2023-04-09 07:54:34,970 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f854bb948a0b48d5830e7d8fcb0195cd 2023-04-09 07:54:34,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/03cd4b2e80a94648bdd256519b80910f 2023-04-09 07:54:34,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/075078c5f5ed46ba99acb19eab365bbb 2023-04-09 07:54:34,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/5849212bf64f43dca380713dbadaecce 2023-04-09 07:54:34,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/57d870393a5549a59cf377345ae62f3a 2023-04-09 07:54:34,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/2062a3ae024a433b90ddf0258f49a415 2023-04-09 07:54:34,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/a91a0e2af2614a50a4d2e05324013ed7 2023-04-09 07:54:34,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/fc19282fb0d944518927b46895e4da1f_SeqId_44_ 2023-04-09 07:54:35,272 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120577e9e586bcb04ba8878ff16869e0b097 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120577e9e586bcb04ba8878ff16869e0b097 2023-04-09 07:54:35,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:35,310 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f27ce2c3b0844d41be6651157e409f99 first=Optional[r00] last=Optional[r2] 2023-04-09 07:54:35,310 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f27ce2c3b0844d41be6651157e409f99}] 2023-04-09 07:54:35,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f27ce2c3b0844d41be6651157e409f99 for inclusion in 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:35,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-09 07:54:35,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:35,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:35,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f27ce2c3b0844d41be6651157e409f99 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__91j07gng851d1abst9ntsvhfpdec99ut4lipqpdek1vilsgu7dfsjmguijn5lb96/family1/f27ce2c3b0844d41be6651157e409f99 2023-04-09 07:54:35,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__91j07gng851d1abst9ntsvhfpdec99ut4lipqpdek1vilsgu7dfsjmguijn5lb96/family1/f27ce2c3b0844d41be6651157e409f99 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_ 2023-04-09 07:54:35,321 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__91j07gng851d1abst9ntsvhfpdec99ut4lipqpdek1vilsgu7dfsjmguijn5lb96/family1/f27ce2c3b0844d41be6651157e409f99 into 33695bd4d17ece9a6c2338030063f345/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_ - updating store file list. 2023-04-09 07:54:35,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_ into 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:35,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__91j07gng851d1abst9ntsvhfpdec99ut4lipqpdek1vilsgu7dfsjmguijn5lb96/family1/f27ce2c3b0844d41be6651157e409f99 into 33695bd4d17ece9a6c2338030063f345/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_) 2023-04-09 07:54:35,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__91j07gng851d1abst9ntsvhfpdec99ut4lipqpdek1vilsgu7dfsjmguijn5lb96/family1/f27ce2c3b0844d41be6651157e409f99 2023-04-09 07:54:35,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:54:35,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:54:35,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 33695bd4d17ece9a6c2338030063f345/family1 is initiating minor compaction (all files) 2023-04-09 07:54:35,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@18312df4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:35,327 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 33695bd4d17ece9a6c2338030063f345/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:35,327 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/c33c2d9133044751a0df596a463495ab, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp, totalSize=390.1 K 2023-04-09 07:54:35,327 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting be99707cc6804cb999c2be5f95260dce, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:54:35,329 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting c33c2d9133044751a0df596a463495ab, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:54:35,330 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840049 2023-04-09 07:54:35,330 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:35,331 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:35,334 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130142171c80aae40e0b5828e7a4d193653 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130142171c80aae40e0b5828e7a4d193653 2023-04-09 07:54:35,336 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512012c2f9597c8394cd58fb3b186b0b6b3a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512012c2f9597c8394cd58fb3b186b0b6b3a7 2023-04-09 07:54:35,342 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4b1256d764e4c90b2baba4a078c3ada to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205f4b1256d764e4c90b2baba4a078c3ada 2023-04-09 07:54:35,342 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-04-09 07:54:35,342 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-09 07:54:35,343 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:35,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:35,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:35,345 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:35,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.5130 sec 2023-04-09 07:54:35,357 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 33695bd4d17ece9a6c2338030063f345#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:35,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-09 07:54:35,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:35,840 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:35,841 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:35,841 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:35,844 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-04-09 07:54:35,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE}] 2023-04-09 07:54:35,848 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE 2023-04-09 07:54:35,848 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:35,849 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026875848"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026875848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026875848"}]},"ts":"1681026875848"} 2023-04-09 07:54:35,850 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:36,002 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 33695bd4d17ece9a6c2338030063f345, disabling compactions & flushes 2023-04-09 07:54:36,002 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,578 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/.tmp/family1/dbd4993d24b2446caff9971dfddfb5ac as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/dbd4993d24b2446caff9971dfddfb5ac 2023-04-09 07:54:36,585 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 33695bd4d17ece9a6c2338030063f345/family1 of 33695bd4d17ece9a6c2338030063f345 into dbd4993d24b2446caff9971dfddfb5ac(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:54:36,585 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:36,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,585 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1681026875327; duration=1sec 2023-04-09 07:54:36,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. after waiting 0 ms 2023-04-09 07:54:36,586 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:36,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,589 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/c33c2d9133044751a0df596a463495ab, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_] to archive 2023-04-09 07:54:36,590 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:36,592 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/be99707cc6804cb999c2be5f95260dce 2023-04-09 07:54:36,595 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/c33c2d9133044751a0df596a463495ab to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/c33c2d9133044751a0df596a463495ab 2023-04-09 07:54:36,597 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f5e2f480e1d94e7e9e4bcbae7eb8eaa2_SeqId_47_ 2023-04-09 07:54:36,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-09 07:54:36,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:36,604 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 33695bd4d17ece9a6c2338030063f345 to self. 2023-04-09 07:54:36,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=CLOSED 2023-04-09 07:54:36,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026876606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026876606"}]},"ts":"1681026876606"} 2023-04-09 07:54:36,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-04-09 07:54:36,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245 in 758 msec 2023-04-09 07:54:36,610 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=true 2023-04-09 07:54:36,760 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:36,761 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026876760"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026876760"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026876760"}]},"ts":"1681026876760"} 2023-04-09 07:54:36,762 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:36,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 33695bd4d17ece9a6c2338030063f345, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:36,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:36,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,927 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,928 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:36,928 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:36,929 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33695bd4d17ece9a6c2338030063f345 columnFamilyName family1 2023-04-09 07:54:36,937 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/dbd4993d24b2446caff9971dfddfb5ac 2023-04-09 07:54:36,937 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(310): Store=33695bd4d17ece9a6c2338030063f345/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:36,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,945 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:36,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 33695bd4d17ece9a6c2338030063f345; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65864023, jitterRate=-0.01854957640171051}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:36,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:36,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., pid=47, masterSystemTime=1681026876914 2023-04-09 07:54:36,950 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:36,950 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026876950"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026876950"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026876950"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026876950"}]},"ts":"1681026876950"} 2023-04-09 07:54:36,952 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:36,953 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-04-09 07:54:36,953 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245 in 190 msec 2023-04-09 07:54:36,957 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-04-09 07:54:36,957 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE in 1.1070 sec 2023-04-09 07:54:36,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-04-09 07:54:36,959 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1130 sec 2023-04-09 07:54:36,961 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5340 sec 2023-04-09 07:54:36,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-04-09 07:54:36,964 INFO [Listener at localhost/35977] compactions.TestMobCompactor(278): alter status finished 2023-04-09 07:54:36,965 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:36,966 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:36,967 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:36,968 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:36,968 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027476968 2023-04-09 07:54:36,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:36,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-09 07:54:36,969 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:36,970 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x423f78b7 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:36,984 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67b0d5c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:36,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:36,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:36,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:36,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:36,984 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,023 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128538b0263b408440cb28049f4456692a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128538b0263b408440cb28049f4456692a7 2023-04-09 07:54:37,024 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:37,039 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:37,040 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54544, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:37,052 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:37,054 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48916, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:37,067 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30bb5e7f66034304b678eb9a102d7697 first=Optional[r05] last=Optional[r8] 2023-04-09 07:54:37,067 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=51 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30bb5e7f66034304b678eb9a102d7697}] 2023-04-09 07:54:37,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30bb5e7f66034304b678eb9a102d7697 for inclusion in 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:37,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-09 07:54:37,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:37,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:37,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/30bb5e7f66034304b678eb9a102d7697 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kke7vmodbhnuavuq7afqigocau5ki7hq72f9fvjo5rl60hbmsj2csn2mdork4dkc/family1/30bb5e7f66034304b678eb9a102d7697 2023-04-09 07:54:37,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kke7vmodbhnuavuq7afqigocau5ki7hq72f9fvjo5rl60hbmsj2csn2mdork4dkc/family1/30bb5e7f66034304b678eb9a102d7697 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f315887a5cbf463c80c586021334d286_SeqId_53_ 2023-04-09 07:54:37,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kke7vmodbhnuavuq7afqigocau5ki7hq72f9fvjo5rl60hbmsj2csn2mdork4dkc/family1/30bb5e7f66034304b678eb9a102d7697 into 33695bd4d17ece9a6c2338030063f345/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f315887a5cbf463c80c586021334d286_SeqId_53_ - updating store file list. 2023-04-09 07:54:37,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f315887a5cbf463c80c586021334d286_SeqId_53_ into 33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:37,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kke7vmodbhnuavuq7afqigocau5ki7hq72f9fvjo5rl60hbmsj2csn2mdork4dkc/family1/30bb5e7f66034304b678eb9a102d7697 into 33695bd4d17ece9a6c2338030063f345/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f315887a5cbf463c80c586021334d286_SeqId_53_) 2023-04-09 07:54:37,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kke7vmodbhnuavuq7afqigocau5ki7hq72f9fvjo5rl60hbmsj2csn2mdork4dkc/family1/30bb5e7f66034304b678eb9a102d7697 2023-04-09 07:54:37,091 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:37,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:37,093 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511205fed0b085e5e4ed1acaca663abb01d2e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511205fed0b085e5e4ed1acaca663abb01d2e 2023-04-09 07:54:37,095 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128316a6aed8c284f78ac425084c9823fdb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128316a6aed8c284f78ac425084c9823fdb 2023-04-09 07:54:37,095 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-04-09 07:54:37,095 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-09 07:54:37,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,095 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:37,095 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:37,097 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:37,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 132 msec 2023-04-09 07:54:37,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:54:37,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:37,174 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-04-09 07:54:37,175 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:37,175 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-04-09 07:54:37,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-04-09 07:54:37,180 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE}] 2023-04-09 07:54:37,181 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE 2023-04-09 07:54:37,182 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:37,182 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026877182"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026877182"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026877182"}]},"ts":"1681026877182"} 2023-04-09 07:54:37,184 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:37,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 33695bd4d17ece9a6c2338030063f345, disabling compactions & flushes 2023-04-09 07:54:37,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. after waiting 0 ms 2023-04-09 07:54:37,337 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,344 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-04-09 07:54:37,345 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:37,345 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 33695bd4d17ece9a6c2338030063f345 to self. 2023-04-09 07:54:37,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,347 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=CLOSED 2023-04-09 07:54:37,348 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026877347"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026877347"}]},"ts":"1681026877347"} 2023-04-09 07:54:37,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-04-09 07:54:37,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245 in 165 msec 2023-04-09 07:54:37,355 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=true 2023-04-09 07:54:37,435 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-09 07:54:37,505 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:37,505 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026877505"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026877505"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026877505"}]},"ts":"1681026877505"} 2023-04-09 07:54:37,507 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:37,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 33695bd4d17ece9a6c2338030063f345, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:37,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:37,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,680 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,682 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:37,682 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1 2023-04-09 07:54:37,682 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33695bd4d17ece9a6c2338030063f345 columnFamilyName family1 2023-04-09 07:54:37,692 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/dbd4993d24b2446caff9971dfddfb5ac 2023-04-09 07:54:37,696 DEBUG [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/family1/f315887a5cbf463c80c586021334d286_SeqId_53_ 2023-04-09 07:54:37,696 INFO [StoreOpener-33695bd4d17ece9a6c2338030063f345-1] regionserver.HStore(310): Store=33695bd4d17ece9a6c2338030063f345/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:37,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:54:37,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 33695bd4d17ece9a6c2338030063f345; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70287536, jitterRate=0.04736590385437012}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:37,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:54:37,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345., pid=53, masterSystemTime=1681026877659 2023-04-09 07:54:37,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:54:37,705 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=33695bd4d17ece9a6c2338030063f345, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:37,705 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1681026877705"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026877705"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026877705"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026877705"}]},"ts":"1681026877705"} 2023-04-09 07:54:37,708 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-04-09 07:54:37,708 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 33695bd4d17ece9a6c2338030063f345, server=jenkins-hbase4.apache.org,34907,1681026816245 in 199 msec 2023-04-09 07:54:37,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-04-09 07:54:37,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=33695bd4d17ece9a6c2338030063f345, REOPEN/MOVE in 528 msec 2023-04-09 07:54:37,711 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-04-09 07:54:37,711 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 533 msec 2023-04-09 07:54:37,713 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 549 msec 2023-04-09 07:54:37,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-09 07:54:37,715 INFO [Listener at localhost/35977] compactions.TestMobCompactor(278): alter status finished 2023-04-09 07:54:37,716 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-04-09 07:54:37,717 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:37,718 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:37,719 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:37,719 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027477719 2023-04-09 07:54:37,720 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:37,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-04-09 07:54:37,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:37,721 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x0df9081b to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:37,727 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c06e295, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:37,727 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,727 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,727 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-04-09 07:54:37,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-04-09 07:54:37,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:37,730 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-04-09 07:54:37,731 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 13 msec 2023-04-09 07:54:37,779 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=253 (was 259), OpenFileDescriptor=626 (was 628), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=338 (was 336) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1644 (was 1729) 2023-04-09 07:54:37,791 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=253, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=338, ProcessCount=169, AvailableMemoryMB=1643 2023-04-09 07:54:37,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-04-09 07:54:37,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:37,795 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:37,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-04-09 07:54:37,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:37,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-09 07:54:37,797 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:37,798 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db empty. 2023-04-09 07:54:37,798 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:37,798 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-04-09 07:54:37,807 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:37,809 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => aa312bcb3ab129175528fadb46d3a2db, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:37,814 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:37,814 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing aa312bcb3ab129175528fadb46d3a2db, disabling compactions & flushes 2023-04-09 07:54:37,814 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:37,814 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:37,815 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. after waiting 0 ms 2023-04-09 07:54:37,815 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:37,815 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:37,815 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:37,817 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:37,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026877818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026877818"}]},"ts":"1681026877818"} 2023-04-09 07:54:37,820 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:54:37,821 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:37,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026877821"}]},"ts":"1681026877821"} 2023-04-09 07:54:37,822 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-04-09 07:54:37,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, ASSIGN}] 2023-04-09 07:54:37,828 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, ASSIGN 2023-04-09 07:54:37,829 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:37,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-09 07:54:37,980 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:37,980 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026877980"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026877980"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026877980"}]},"ts":"1681026877980"} 2023-04-09 07:54:37,983 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:38,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-09 07:54:38,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:38,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aa312bcb3ab129175528fadb46d3a2db, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:38,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:38,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,140 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,145 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:38,146 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:38,146 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa312bcb3ab129175528fadb46d3a2db columnFamilyName family1 2023-04-09 07:54:38,147 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(310): Store=aa312bcb3ab129175528fadb46d3a2db/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:38,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:38,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:38,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aa312bcb3ab129175528fadb46d3a2db; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73856775, jitterRate=0.10055170953273773}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:38,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:38,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., pid=57, masterSystemTime=1681026878135 2023-04-09 07:54:38,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:38,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:38,160 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:38,160 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026878160"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026878160"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026878160"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026878160"}]},"ts":"1681026878160"} 2023-04-09 07:54:38,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-04-09 07:54:38,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245 in 178 msec 2023-04-09 07:54:38,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-04-09 07:54:38,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, ASSIGN in 337 msec 2023-04-09 07:54:38,165 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:38,166 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026878165"}]},"ts":"1681026878165"} 2023-04-09 07:54:38,167 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-04-09 07:54:38,170 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:38,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 377 msec 2023-04-09 07:54:38,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-04-09 07:54:38,404 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-04-09 07:54:38,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:38,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,446 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:38,446 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:38,446 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:38,446 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:38,446 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,447 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,448 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:38,448 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:38,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:38,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:38,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,449 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:38,449 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:38,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:38,450 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:38,452 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:38,452 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:38,452 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:38,452 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:38,453 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:54:38,923 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307295cd7a174b462ebc3eec3014357660 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511307295cd7a174b462ebc3eec3014357660 2023-04-09 07:54:39,327 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/69e2aa4d5e6d427f947d8859b0b8721c 2023-04-09 07:54:39,332 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/69e2aa4d5e6d427f947d8859b0b8721c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c 2023-04-09 07:54:39,335 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c, entries=1000, sequenceid=5, filesize=190.5 K 2023-04-09 07:54:39,336 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 883ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:39,336 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:39,337 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:39,337 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:39,337 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:39,337 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:39,337 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,337 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:39,337 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:39,338 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,338 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:39,339 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,339 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:39,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:39,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:39,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53354a4d[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:39,340 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:39,340 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,345 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,345 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:39,345 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,345 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:39,345 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:39,345 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:39,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:39,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:39,346 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,346 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:39,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:39,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:39,349 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:39,349 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:39,349 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:39,349 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,349 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:39,350 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,350 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:39,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,350 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:39,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:39,351 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:39,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:39,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:39,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:39,351 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:39,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,357 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:39,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:39,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:39,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:39,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:39,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:39,357 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:39,358 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:39,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,358 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:39,458 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:39,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:39,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:39,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:39,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,464 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:39,464 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:39,464 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:39,464 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:39,465 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,465 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,466 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:39,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:39,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:39,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:39,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:39,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,467 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:39,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:39,467 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:39,467 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:39,467 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:54:39,879 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d359f87040414ec8a988d70abdde3416 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128d359f87040414ec8a988d70abdde3416 2023-04-09 07:54:40,284 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/186caf29b3c64cacbac80436e00b06e0 2023-04-09 07:54:40,289 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/186caf29b3c64cacbac80436e00b06e0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0 2023-04-09 07:54:40,292 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:54:40,293 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 826ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:40,293 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:40,293 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:40,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:40,295 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,295 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:40,295 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,295 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:40,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:40,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:40,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:40,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6dd9e010[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:40,296 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:40,296 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,297 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:40,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:40,297 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,297 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:40,297 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:40,298 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:40,298 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:40,299 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:40,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:40,301 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:40,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:40,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:40,301 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,301 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:40,303 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,303 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:40,303 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:40,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:40,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:40,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:40,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,305 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:40,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:40,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:40,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:40,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:40,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:40,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,307 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:40,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:40,307 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:40,407 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:40,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:40,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:40,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:40,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,414 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:40,414 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:40,414 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:40,414 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:40,414 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,414 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,415 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,415 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:40,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:40,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:40,416 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:40,416 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,416 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:40,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,416 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,417 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:40,417 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:40,417 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:40,417 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:40,417 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:40,417 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:40,417 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:40,417 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:40,417 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:40,417 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:40,835 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120173c58f6d2f094c54af44ee6f1fc4b9f4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120173c58f6d2f094c54af44ee6f1fc4b9f4 2023-04-09 07:54:41,240 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/09b44ecbdf554ed99b080b0fc00b0c2b 2023-04-09 07:54:41,245 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/09b44ecbdf554ed99b080b0fc00b0c2b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b 2023-04-09 07:54:41,248 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b, entries=1, sequenceid=13, filesize=5.0 K 2023-04-09 07:54:41,249 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 832ms, sequenceid=13, compaction requested=true 2023-04-09 07:54:41,249 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:41,249 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:41,249 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,250 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:41,250 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:41,251 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:41,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:41,251 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,252 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:41,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:41,253 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f22053f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:41,253 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:41,253 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,254 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,255 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:41,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,255 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:41,255 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:41,255 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:41,256 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:41,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:41,258 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:41,258 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:41,258 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:41,258 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,258 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:41,259 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,259 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,259 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:41,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:41,260 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:41,260 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,262 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,262 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:41,262 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:41,262 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:41,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:41,262 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:41,262 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,262 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:41,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,263 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:41,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:41,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:41,363 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:41,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:41,367 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:41,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:41,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,369 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:41,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:41,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:41,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:41,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,370 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,372 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,372 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:41,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:41,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:41,372 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:41,372 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:41,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,372 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:41,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:41,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:41,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:41,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:41,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:41,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:41,373 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:41,373 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:41,373 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:41,373 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:41,786 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e61906b731cd44ff9145a83ec1f892bb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61906b731cd44ff9145a83ec1f892bb 2023-04-09 07:54:42,106 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:42,191 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/56c32ae54da14eb5950a9e87e16b3e50 2023-04-09 07:54:42,196 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/56c32ae54da14eb5950a9e87e16b3e50 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50 2023-04-09 07:54:42,200 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50, entries=1, sequenceid=17, filesize=5.0 K 2023-04-09 07:54:42,200 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 827ms, sequenceid=17, compaction requested=true 2023-04-09 07:54:42,201 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:42,201 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:42,201 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:42,201 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:42,201 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,201 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:42,201 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:42,202 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,202 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:42,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:42,202 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,202 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:42,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:42,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22b2c714[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:42,204 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:42,204 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,208 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,208 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:42,208 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:42,208 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:42,208 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,208 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:42,209 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:42,210 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:42,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:42,212 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:42,212 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:42,212 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:42,212 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,212 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:42,213 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,213 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:42,213 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,214 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:42,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,215 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:42,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:42,218 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:42,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:42,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:42,218 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,218 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:42,219 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:42,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:42,319 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:42,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:42,323 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:42,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:42,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,325 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:42,325 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:42,325 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:42,325 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:42,326 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,326 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,327 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,327 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:42,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:42,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:42,327 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:42,327 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:42,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:42,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:42,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:42,328 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:42,328 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:42,328 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:42,328 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:42,328 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:42,328 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:42,328 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:42,329 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:42,740 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ee50056357ad47d2a5397af3343b67fb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ee50056357ad47d2a5397af3343b67fb 2023-04-09 07:54:43,145 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/cf309cc6348b4c03858db43f7a6b8670 2023-04-09 07:54:43,150 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/cf309cc6348b4c03858db43f7a6b8670 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670 2023-04-09 07:54:43,153 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670, entries=1, sequenceid=21, filesize=5.0 K 2023-04-09 07:54:43,154 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 825ms, sequenceid=21, compaction requested=true 2023-04-09 07:54:43,154 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:43,154 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:43,154 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:43,154 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:43,154 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,154 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:43,154 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:43,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,156 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:43,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:43,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bac862d[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:43,157 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:43,157 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,158 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:43,158 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:43,158 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,158 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:43,160 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:43,160 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,160 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:43,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,160 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:43,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:43,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:43,162 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:43,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:43,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:43,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,162 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:43,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:43,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:43,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:43,164 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:43,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,165 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:43,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:43,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,167 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:43,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:43,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,167 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:43,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:43,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:43,168 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,168 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,268 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:43,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:43,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:43,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:43,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,273 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:43,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:43,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:43,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:43,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,275 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,275 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:43,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:43,275 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:43,275 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,276 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:43,276 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:43,277 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:43,277 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:43,277 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:43,688 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123180040a0b0ff640d4b07aeeb398d90e4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123180040a0b0ff640d4b07aeeb398d90e4f 2023-04-09 07:54:44,093 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/fe03f58945484f26bb04f3f68df3f89d 2023-04-09 07:54:44,097 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/fe03f58945484f26bb04f3f68df3f89d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d 2023-04-09 07:54:44,101 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d, entries=1, sequenceid=25, filesize=5.0 K 2023-04-09 07:54:44,102 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 825ms, sequenceid=25, compaction requested=true 2023-04-09 07:54:44,102 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:44,102 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:44,102 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:44,103 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,103 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:44,104 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,104 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:44,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:44,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:44,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79d73428[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:44,105 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:44,105 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,106 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:44,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:44,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:44,106 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,106 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:44,107 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,107 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:44,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:44,107 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:44,107 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:44,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:44,109 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:44,109 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:44,109 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:44,109 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,109 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:44,111 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,111 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:44,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:44,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,113 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:44,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:44,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:44,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:44,119 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:44,119 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,120 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:44,220 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:44,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:44,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:44,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:44,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,226 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:44,226 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:44,226 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:44,226 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:44,226 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,226 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,232 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:44,232 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:44,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:44,232 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:44,232 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:44,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:44,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:44,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:44,233 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:44,233 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:44,233 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:44,233 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:44,645 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e230fdbb51ea46f489a7899aba915383 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e230fdbb51ea46f489a7899aba915383 2023-04-09 07:54:45,050 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4 2023-04-09 07:54:45,054 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4 2023-04-09 07:54:45,058 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4, entries=1, sequenceid=29, filesize=5.0 K 2023-04-09 07:54:45,058 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 825ms, sequenceid=29, compaction requested=true 2023-04-09 07:54:45,058 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:45,058 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:45,058 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:45,058 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:45,059 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,059 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:45,059 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:45,060 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,060 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:45,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:45,060 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,060 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:45,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:45,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6643ddd0[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:45,062 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:45,062 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,064 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,064 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:45,064 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:45,065 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:45,065 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,065 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:45,066 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:45,066 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:45,066 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,066 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:45,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:45,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:45,068 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:45,068 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:45,068 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:45,068 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,068 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:45,069 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,069 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:45,069 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:45,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:45,070 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,071 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:45,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:45,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,073 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:45,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:45,074 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:45,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:45,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:45,174 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:45,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:45,178 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:45,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:45,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,179 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:45,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:45,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:45,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:45,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,182 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:45,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:45,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:45,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:45,183 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:45,183 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:45,183 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:45,183 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:45,183 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:45,183 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:45,183 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:45,183 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:45,595 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036f3fb904cb734d53b25b5cf883d016a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601036f3fb904cb734d53b25b5cf883d016a7 2023-04-09 07:54:46,000 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/9c38f85b20b24817a78a08f2cfc4535b 2023-04-09 07:54:46,005 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/9c38f85b20b24817a78a08f2cfc4535b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b 2023-04-09 07:54:46,009 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b, entries=1, sequenceid=33, filesize=5.0 K 2023-04-09 07:54:46,009 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 826ms, sequenceid=33, compaction requested=true 2023-04-09 07:54:46,009 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:46,009 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:46,009 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:46,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:46,012 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,012 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,012 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,012 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:46,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:46,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@527d3c6b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:46,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:46,014 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,014 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,015 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:46,015 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:46,015 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:46,015 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,015 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:46,016 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:46,016 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:46,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:46,018 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:46,018 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:46,018 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:46,018 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,018 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:46,019 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,019 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,019 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,019 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,021 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:46,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:46,024 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:46,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:46,024 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:46,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:46,024 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:46,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:46,124 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:46,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:46,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:46,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:46,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,131 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:46,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:46,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:46,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:46,132 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,132 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:46,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:46,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:46,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:46,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:46,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,134 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:46,134 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,134 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:46,134 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:46,134 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:46,134 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:46,134 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:46,134 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:46,134 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:46,135 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:46,153 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907dcb3e5dade6041acb49eaee844b903cc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907dcb3e5dade6041acb49eaee844b903cc 2023-04-09 07:54:46,164 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/447355f43a224363b7c54068a58b1297 2023-04-09 07:54:46,169 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/447355f43a224363b7c54068a58b1297 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297 2023-04-09 07:54:46,173 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297, entries=1, sequenceid=37, filesize=5.0 K 2023-04-09 07:54:46,174 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 39ms, sequenceid=37, compaction requested=true 2023-04-09 07:54:46,174 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:46,175 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:46,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,176 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:46,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ddf55e7[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:46,178 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:46,178 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,180 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:46,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:46,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,180 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:46,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:46,182 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:46,182 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:46,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:46,189 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:46,189 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:46,189 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:46,189 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:46,189 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,189 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:46,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:46,191 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:46,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:46,191 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:46,193 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:46,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,197 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,197 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:46,197 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:46,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:46,197 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:46,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:46,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:46,197 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:46,197 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:46,198 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:46,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:46,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:46,298 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:46,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:46,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:46,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:46,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,304 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:46,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:46,304 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-04-09 07:54:46,305 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-04-09 07:54:46,305 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,305 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,307 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,307 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:46,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:46,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:46,307 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:46,308 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:46,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-04-09 07:54:46,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:46,309 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-04-09 07:54:46,309 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:46,309 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-04-09 07:54:46,309 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:54:46,309 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:46,309 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:46,309 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. started... 2023-04-09 07:54:46,309 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing aa312bcb3ab129175528fadb46d3a2db 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:54:46,722 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c451ffb114684ab0b4bd5863bb919fb4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c451ffb114684ab0b4bd5863bb919fb4 2023-04-09 07:54:47,133 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/01384840716a4a7887ba498162fadcb3 2023-04-09 07:54:47,139 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/01384840716a4a7887ba498162fadcb3 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3 2023-04-09 07:54:47,143 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3, entries=1, sequenceid=41, filesize=5.0 K 2023-04-09 07:54:47,144 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa312bcb3ab129175528fadb46d3a2db in 835ms, sequenceid=41, compaction requested=true 2023-04-09 07:54:47,144 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:47,145 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-04-09 07:54:47,145 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-04-09 07:54:47,146 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,146 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:47,147 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:47,147 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-04-09 07:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2de94a00[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:47,148 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-04-09 07:54:47,148 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,149 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,149 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-04-09 07:54:47,149 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-04-09 07:54:47,150 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:47,150 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,150 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:47,151 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-04-09 07:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:47,151 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-04-09 07:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-04-09 07:54:47,154 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-04-09 07:54:47,154 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:47,154 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:47,154 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,154 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:47,157 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,157 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,157 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:47,157 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,158 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:47,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-04-09 07:54:47,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-04-09 07:54:47,161 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:47,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:47,161 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:47,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:47,262 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:47,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:47,264 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:47,266 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:47,267 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:47,267 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027487267 2023-04-09 07:54:47,268 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:47,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:47,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-09 07:54:47,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:47,276 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x6c048f43 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:47,281 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@353c96a0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,757 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039535ddb08b77437eb8e20cdf584b4dea to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601039535ddb08b77437eb8e20cdf584b4dea 2023-04-09 07:54:47,758 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:47,780 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:47,782 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49336, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:47,793 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:47,794 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55302, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:47,822 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d936451b58348bbbe43e24e647f41cd first=Optional[r3] last=Optional[r6] 2023-04-09 07:54:47,823 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d936451b58348bbbe43e24e647f41cd}] 2023-04-09 07:54:47,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d936451b58348bbbe43e24e647f41cd for inclusion in aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:47,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-04-09 07:54:47,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:47,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:47,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3d936451b58348bbbe43e24e647f41cd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2r3elt9mvrrab4q9gam8qvcg59j10okpgngfcgm1rcmb93m5vjb91lo6j1st3on/family1/3d936451b58348bbbe43e24e647f41cd 2023-04-09 07:54:47,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2r3elt9mvrrab4q9gam8qvcg59j10okpgngfcgm1rcmb93m5vjb91lo6j1st3on/family1/3d936451b58348bbbe43e24e647f41cd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_ 2023-04-09 07:54:47,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2r3elt9mvrrab4q9gam8qvcg59j10okpgngfcgm1rcmb93m5vjb91lo6j1st3on/family1/3d936451b58348bbbe43e24e647f41cd into aa312bcb3ab129175528fadb46d3a2db/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_ - updating store file list. 2023-04-09 07:54:47,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_ into aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:47,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2r3elt9mvrrab4q9gam8qvcg59j10okpgngfcgm1rcmb93m5vjb91lo6j1st3on/family1/3d936451b58348bbbe43e24e647f41cd into aa312bcb3ab129175528fadb46d3a2db/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_) 2023-04-09 07:54:47,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__v2r3elt9mvrrab4q9gam8qvcg59j10okpgngfcgm1rcmb93m5vjb91lo6j1st3on/family1/3d936451b58348bbbe43e24e647f41cd 2023-04-09 07:54:47,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:54:47,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:54:47,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): aa312bcb3ab129175528fadb46d3a2db/family1 is initiating minor compaction 2023-04-09 07:54:47,860 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aa312bcb3ab129175528fadb46d3a2db/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:47,860 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp, totalSize=50.3 K 2023-04-09 07:54:47,860 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 186caf29b3c64cacbac80436e00b06e0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 09b44ecbdf554ed99b080b0fc00b0c2b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 56c32ae54da14eb5950a9e87e16b3e50, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting cf309cc6348b4c03858db43f7a6b8670, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting fe03f58945484f26bb04f3f68df3f89d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 87cd2ee39d6d41a59a3b4d9a4c3ba3c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 9c38f85b20b24817a78a08f2cfc4535b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 447355f43a224363b7c54068a58b1297, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 01384840716a4a7887ba498162fadcb3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:54:47,861 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 6049a1eb44f642fe8bb26826b452edec_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:54:47,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@75e93031; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:47,864 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:47,865 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:47,867 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ee50056357ad47d2a5397af3343b67fb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228ee50056357ad47d2a5397af3343b67fb 2023-04-09 07:54:47,870 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123180040a0b0ff640d4b07aeeb398d90e4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123180040a0b0ff640d4b07aeeb398d90e4f 2023-04-09 07:54:47,871 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e230fdbb51ea46f489a7899aba915383 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e230fdbb51ea46f489a7899aba915383 2023-04-09 07:54:47,873 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601036f3fb904cb734d53b25b5cf883d016a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601036f3fb904cb734d53b25b5cf883d016a7 2023-04-09 07:54:47,873 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-04-09 07:54:47,873 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-04-09 07:54:47,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,874 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:47,944 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa312bcb3ab129175528fadb46d3a2db#family1#compaction#72 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:48,369 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059594778977b44f89b9b5a62869302a43 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512059594778977b44f89b9b5a62869302a43 2023-04-09 07:54:48,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:48,383 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/7d48a84f5629409db44fa06a6619ae6d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/7d48a84f5629409db44fa06a6619ae6d 2023-04-09 07:54:48,390 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in aa312bcb3ab129175528fadb46d3a2db/family1 of aa312bcb3ab129175528fadb46d3a2db into 7d48a84f5629409db44fa06a6619ae6d(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:48,391 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:48,391 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1681026887860; duration=0sec 2023-04-09 07:54:48,391 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:48,406 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/78b321e249ef46acb67c296fad0aa71a first=Optional[r00] last=Optional[r2] 2023-04-09 07:54:48,406 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/78b321e249ef46acb67c296fad0aa71a}] 2023-04-09 07:54:48,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/78b321e249ef46acb67c296fad0aa71a for inclusion in aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:48,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-04-09 07:54:48,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:48,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:48,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/78b321e249ef46acb67c296fad0aa71a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9a1ql156h26fauknjqm5qunmf6gfotsb4arlb108v06g0rtk36mpt26evd37vs04/family1/78b321e249ef46acb67c296fad0aa71a 2023-04-09 07:54:48,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9a1ql156h26fauknjqm5qunmf6gfotsb4arlb108v06g0rtk36mpt26evd37vs04/family1/78b321e249ef46acb67c296fad0aa71a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_ 2023-04-09 07:54:48,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9a1ql156h26fauknjqm5qunmf6gfotsb4arlb108v06g0rtk36mpt26evd37vs04/family1/78b321e249ef46acb67c296fad0aa71a into aa312bcb3ab129175528fadb46d3a2db/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_ - updating store file list. 2023-04-09 07:54:48,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_ into aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:48,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9a1ql156h26fauknjqm5qunmf6gfotsb4arlb108v06g0rtk36mpt26evd37vs04/family1/78b321e249ef46acb67c296fad0aa71a into aa312bcb3ab129175528fadb46d3a2db/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_) 2023-04-09 07:54:48,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__9a1ql156h26fauknjqm5qunmf6gfotsb4arlb108v06g0rtk36mpt26evd37vs04/family1/78b321e249ef46acb67c296fad0aa71a 2023-04-09 07:54:48,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:54:48,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:54:48,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): aa312bcb3ab129175528fadb46d3a2db/family1 is initiating minor compaction (all files) 2023-04-09 07:54:48,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@767d9f08; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:48,421 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of aa312bcb3ab129175528fadb46d3a2db/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:48,421 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/7d48a84f5629409db44fa06a6619ae6d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp, totalSize=388.1 K 2023-04-09 07:54:48,421 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 69e2aa4d5e6d427f947d8859b0b8721c, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:54:48,421 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 7d48a84f5629409db44fa06a6619ae6d, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:54:48,422 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 15d47abab1744bc4b77ac06b71adfabb_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840049 2023-04-09 07:54:48,423 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:48,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:48,425 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511307295cd7a174b462ebc3eec3014357660 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511307295cd7a174b462ebc3eec3014357660 2023-04-09 07:54:48,426 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120173c58f6d2f094c54af44ee6f1fc4b9f4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120173c58f6d2f094c54af44ee6f1fc4b9f4 2023-04-09 07:54:48,428 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61906b731cd44ff9145a83ec1f892bb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205e61906b731cd44ff9145a83ec1f892bb 2023-04-09 07:54:48,428 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-04-09 07:54:48,428 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-04-09 07:54:48,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:48,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:48,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:48,430 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:48,431 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.1650 sec 2023-04-09 07:54:48,437 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa312bcb3ab129175528fadb46d3a2db#family1#compaction#73 average throughput is 25.66 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:48,494 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_] to archive 2023-04-09 07:54:48,496 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:48,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/186caf29b3c64cacbac80436e00b06e0 2023-04-09 07:54:48,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/09b44ecbdf554ed99b080b0fc00b0c2b 2023-04-09 07:54:48,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/56c32ae54da14eb5950a9e87e16b3e50 2023-04-09 07:54:48,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/cf309cc6348b4c03858db43f7a6b8670 2023-04-09 07:54:48,505 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/.tmp/family1/a6c9a33a8e70479494fe7596fd7e4a27 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/a6c9a33a8e70479494fe7596fd7e4a27 2023-04-09 07:54:48,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/fe03f58945484f26bb04f3f68df3f89d 2023-04-09 07:54:48,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/87cd2ee39d6d41a59a3b4d9a4c3ba3c4 2023-04-09 07:54:48,513 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aa312bcb3ab129175528fadb46d3a2db/family1 of aa312bcb3ab129175528fadb46d3a2db into a6c9a33a8e70479494fe7596fd7e4a27(size=192.4 K), total size for store is 192.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:48,513 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:48,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/9c38f85b20b24817a78a08f2cfc4535b 2023-04-09 07:54:48,513 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1681026888420; duration=0sec 2023-04-09 07:54:48,514 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:48,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/447355f43a224363b7c54068a58b1297 2023-04-09 07:54:48,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/01384840716a4a7887ba498162fadcb3 2023-04-09 07:54:48,518 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/6049a1eb44f642fe8bb26826b452edec_SeqId_44_ 2023-04-09 07:54:48,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-09 07:54:48,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:48,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/7d48a84f5629409db44fa06a6619ae6d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_] to archive 2023-04-09 07:54:48,551 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:48,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/69e2aa4d5e6d427f947d8859b0b8721c 2023-04-09 07:54:48,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/7d48a84f5629409db44fa06a6619ae6d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/7d48a84f5629409db44fa06a6619ae6d 2023-04-09 07:54:48,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/15d47abab1744bc4b77ac06b71adfabb_SeqId_47_ 2023-04-09 07:54:48,958 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:48,960 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:48,960 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:48,962 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-04-09 07:54:48,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE}] 2023-04-09 07:54:48,965 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE 2023-04-09 07:54:48,966 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:48,966 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026888966"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026888966"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026888966"}]},"ts":"1681026888966"} 2023-04-09 07:54:48,967 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:49,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aa312bcb3ab129175528fadb46d3a2db, disabling compactions & flushes 2023-04-09 07:54:49,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. after waiting 0 ms 2023-04-09 07:54:49,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-09 07:54:49,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:49,128 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: aa312bcb3ab129175528fadb46d3a2db to self. 2023-04-09 07:54:49,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,130 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=CLOSED 2023-04-09 07:54:49,130 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026889130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026889130"}]},"ts":"1681026889130"} 2023-04-09 07:54:49,133 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-04-09 07:54:49,133 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245 in 164 msec 2023-04-09 07:54:49,133 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=true 2023-04-09 07:54:49,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:49,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026889284"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026889284"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026889284"}]},"ts":"1681026889284"} 2023-04-09 07:54:49,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:49,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aa312bcb3ab129175528fadb46d3a2db, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:49,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:49,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,444 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,446 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:49,446 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:49,446 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa312bcb3ab129175528fadb46d3a2db columnFamilyName family1 2023-04-09 07:54:49,451 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/a6c9a33a8e70479494fe7596fd7e4a27 2023-04-09 07:54:49,451 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(310): Store=aa312bcb3ab129175528fadb46d3a2db/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:49,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:49,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aa312bcb3ab129175528fadb46d3a2db; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61750508, jitterRate=-0.07984572649002075}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:49,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:49,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., pid=63, masterSystemTime=1681026889439 2023-04-09 07:54:49,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:49,460 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:49,460 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026889460"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026889460"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026889460"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026889460"}]},"ts":"1681026889460"} 2023-04-09 07:54:49,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-04-09 07:54:49,463 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245 in 175 msec 2023-04-09 07:54:49,464 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-04-09 07:54:49,465 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE in 499 msec 2023-04-09 07:54:49,468 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-04-09 07:54:49,468 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 503 msec 2023-04-09 07:54:49,473 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 930 msec 2023-04-09 07:54:49,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-04-09 07:54:49,475 INFO [Listener at localhost/35977] compactions.TestMobCompactor(278): alter status finished 2023-04-09 07:54:49,476 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:49,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:49,478 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:49,479 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:49,479 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027489479 2023-04-09 07:54:49,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:49,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-09 07:54:49,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:49,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x5b6cade2 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:49,503 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b14d1de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:49,503 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:49,503 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:49,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:49,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:49,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:49,933 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128882b9fc9e35640308de5acb1e4fd6461 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128882b9fc9e35640308de5acb1e4fd6461 2023-04-09 07:54:49,934 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:49,947 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:49,948 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49340, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:49,954 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:49,956 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55306, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:49,973 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b04d81f87bda42dc86aba12197dcd153 first=Optional[r05] last=Optional[r8] 2023-04-09 07:54:49,974 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=51 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b04d81f87bda42dc86aba12197dcd153}] 2023-04-09 07:54:49,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b04d81f87bda42dc86aba12197dcd153 for inclusion in aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:49,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-04-09 07:54:49,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:49,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:49,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b04d81f87bda42dc86aba12197dcd153 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f3umne5f8a21377t5vvudujv83v4kicakkl1bcd5tfb8n6jhgqe29j7e7pt4na2u/family1/b04d81f87bda42dc86aba12197dcd153 2023-04-09 07:54:49,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f3umne5f8a21377t5vvudujv83v4kicakkl1bcd5tfb8n6jhgqe29j7e7pt4na2u/family1/b04d81f87bda42dc86aba12197dcd153 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/11e98af715644e919d9156ed37decc60_SeqId_53_ 2023-04-09 07:54:49,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f3umne5f8a21377t5vvudujv83v4kicakkl1bcd5tfb8n6jhgqe29j7e7pt4na2u/family1/b04d81f87bda42dc86aba12197dcd153 into aa312bcb3ab129175528fadb46d3a2db/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/11e98af715644e919d9156ed37decc60_SeqId_53_ - updating store file list. 2023-04-09 07:54:50,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/11e98af715644e919d9156ed37decc60_SeqId_53_ into aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:50,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f3umne5f8a21377t5vvudujv83v4kicakkl1bcd5tfb8n6jhgqe29j7e7pt4na2u/family1/b04d81f87bda42dc86aba12197dcd153 into aa312bcb3ab129175528fadb46d3a2db/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/11e98af715644e919d9156ed37decc60_SeqId_53_) 2023-04-09 07:54:50,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__f3umne5f8a21377t5vvudujv83v4kicakkl1bcd5tfb8n6jhgqe29j7e7pt4na2u/family1/b04d81f87bda42dc86aba12197dcd153 2023-04-09 07:54:50,005 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:50,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:50,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c451ffb114684ab0b4bd5863bb919fb4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120c451ffb114684ab0b4bd5863bb919fb4 2023-04-09 07:54:50,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128d359f87040414ec8a988d70abdde3416 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128d359f87040414ec8a988d70abdde3416 2023-04-09 07:54:50,011 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-04-09 07:54:50,011 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-09 07:54:50,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:50,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:50,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:50,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:50,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:50,013 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:50,015 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 536 msec 2023-04-09 07:54:50,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-04-09 07:54:50,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:50,484 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-04-09 07:54:50,485 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-04-09 07:54:50,485 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-04-09 07:54:50,487 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-04-09 07:54:50,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE}] 2023-04-09 07:54:50,490 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE 2023-04-09 07:54:50,491 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:50,491 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026890491"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026890491"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026890491"}]},"ts":"1681026890491"} 2023-04-09 07:54:50,492 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:50,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aa312bcb3ab129175528fadb46d3a2db, disabling compactions & flushes 2023-04-09 07:54:50,644 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. after waiting 0 ms 2023-04-09 07:54:50,644 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,656 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-04-09 07:54:50,657 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,657 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:50,657 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: aa312bcb3ab129175528fadb46d3a2db to self. 2023-04-09 07:54:50,658 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,659 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=CLOSED 2023-04-09 07:54:50,659 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026890659"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026890659"}]},"ts":"1681026890659"} 2023-04-09 07:54:50,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-04-09 07:54:50,662 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245 in 168 msec 2023-04-09 07:54:50,663 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=true 2023-04-09 07:54:50,814 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:50,814 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026890813"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026890813"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026890813"}]},"ts":"1681026890813"} 2023-04-09 07:54:50,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:50,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => aa312bcb3ab129175528fadb46d3a2db, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,973 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,975 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:50,975 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1 2023-04-09 07:54:50,975 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa312bcb3ab129175528fadb46d3a2db columnFamilyName family1 2023-04-09 07:54:50,981 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/11e98af715644e919d9156ed37decc60_SeqId_53_ 2023-04-09 07:54:50,985 DEBUG [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/family1/a6c9a33a8e70479494fe7596fd7e4a27 2023-04-09 07:54:50,985 INFO [StoreOpener-aa312bcb3ab129175528fadb46d3a2db-1] regionserver.HStore(310): Store=aa312bcb3ab129175528fadb46d3a2db/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:50,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:54:50,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened aa312bcb3ab129175528fadb46d3a2db; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60740652, jitterRate=-0.09489375352859497}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:50,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:54:50,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db., pid=69, masterSystemTime=1681026890968 2023-04-09 07:54:50,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:54:50,995 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=aa312bcb3ab129175528fadb46d3a2db, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:50,995 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1681026890995"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026890995"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026890995"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026890995"}]},"ts":"1681026890995"} 2023-04-09 07:54:50,998 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-04-09 07:54:50,998 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure aa312bcb3ab129175528fadb46d3a2db, server=jenkins-hbase4.apache.org,34907,1681026816245 in 180 msec 2023-04-09 07:54:50,999 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-04-09 07:54:50,999 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=aa312bcb3ab129175528fadb46d3a2db, REOPEN/MOVE in 509 msec 2023-04-09 07:54:51,001 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-04-09 07:54:51,001 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 513 msec 2023-04-09 07:54:51,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 929 msec 2023-04-09 07:54:51,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-04-09 07:54:51,005 INFO [Listener at localhost/35977] compactions.TestMobCompactor(278): alter status finished 2023-04-09 07:54:51,007 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-04-09 07:54:51,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:51,009 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:51,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:51,013 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027491013 2023-04-09 07:54:51,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:54:51,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-04-09 07:54:51,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:51,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x342c8467 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:51,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@25b7012c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:51,020 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-04-09 07:54:51,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-04-09 07:54:51,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:51,022 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-04-09 07:54:51,023 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 15 msec 2023-04-09 07:54:51,086 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=256 (was 253) - Thread LEAK? -, OpenFileDescriptor=650 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=294 (was 338), ProcessCount=169 (was 169), AvailableMemoryMB=1636 (was 1643) 2023-04-09 07:54:51,099 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=256, OpenFileDescriptor=650, MaxFileDescriptor=60000, SystemLoadAverage=294, ProcessCount=169, AvailableMemoryMB=1635 2023-04-09 07:54:51,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-09 07:54:51,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:51,105 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:51,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-04-09 07:54:51,106 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:51,109 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,110 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 empty. 2023-04-09 07:54:51,110 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,110 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-04-09 07:54:51,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-09 07:54:51,126 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:51,129 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 438a987d05d3cb50118ccd8c8e13e0c8, NAME => 'testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:51,144 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:51,145 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 438a987d05d3cb50118ccd8c8e13e0c8, disabling compactions & flushes 2023-04-09 07:54:51,145 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,145 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,145 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. after waiting 0 ms 2023-04-09 07:54:51,145 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,145 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,145 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:51,147 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:51,148 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1681026891148"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026891148"}]},"ts":"1681026891148"} 2023-04-09 07:54:51,156 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:54:51,157 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:51,158 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026891157"}]},"ts":"1681026891157"} 2023-04-09 07:54:51,159 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-04-09 07:54:51,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, ASSIGN}] 2023-04-09 07:54:51,165 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, ASSIGN 2023-04-09 07:54:51,166 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:51,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-09 07:54:51,317 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=438a987d05d3cb50118ccd8c8e13e0c8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:51,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1681026891317"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026891317"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026891317"}]},"ts":"1681026891317"} 2023-04-09 07:54:51,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 438a987d05d3cb50118ccd8c8e13e0c8, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:51,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-09 07:54:51,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 438a987d05d3cb50118ccd8c8e13e0c8, NAME => 'testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:54:51,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-04-09 07:54:51,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-04-09 07:54:51,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:51,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,478 INFO [StoreOpener-438a987d05d3cb50118ccd8c8e13e0c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,481 DEBUG [StoreOpener-438a987d05d3cb50118ccd8c8e13e0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1 2023-04-09 07:54:51,481 DEBUG [StoreOpener-438a987d05d3cb50118ccd8c8e13e0c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1 2023-04-09 07:54:51,482 INFO [StoreOpener-438a987d05d3cb50118ccd8c8e13e0c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 438a987d05d3cb50118ccd8c8e13e0c8 columnFamilyName f1 2023-04-09 07:54:51,484 INFO [StoreOpener-438a987d05d3cb50118ccd8c8e13e0c8-1] regionserver.HStore(310): Store=438a987d05d3cb50118ccd8c8e13e0c8/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:51,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:51,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:51,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 438a987d05d3cb50118ccd8c8e13e0c8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68652701, jitterRate=0.023004963994026184}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:54:51,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:51,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8., pid=73, masterSystemTime=1681026891470 2023-04-09 07:54:51,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:51,502 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=438a987d05d3cb50118ccd8c8e13e0c8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:51,502 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1681026891502"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026891502"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026891502"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026891502"}]},"ts":"1681026891502"} 2023-04-09 07:54:51,506 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-04-09 07:54:51,507 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 438a987d05d3cb50118ccd8c8e13e0c8, server=jenkins-hbase4.apache.org,34907,1681026816245 in 185 msec 2023-04-09 07:54:51,509 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-04-09 07:54:51,509 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, ASSIGN in 343 msec 2023-04-09 07:54:51,509 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:51,510 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026891510"}]},"ts":"1681026891510"} 2023-04-09 07:54:51,511 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-04-09 07:54:51,513 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:51,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 410 msec 2023-04-09 07:54:51,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-04-09 07:54:51,717 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-04-09 07:54:51,717 DEBUG [Listener at localhost/35977] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-04-09 07:54:51,719 INFO [Listener at localhost/35977] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-04-09 07:54:51,722 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-04-09 07:54:51,723 INFO [Listener at localhost/35977] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-04-09 07:54:51,723 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-04-09 07:54:51,736 INFO [Listener at localhost/35977] regionserver.HRegion(2745): Flushing 438a987d05d3cb50118ccd8c8e13e0c8 1/1 column families, dataSize=37 B heapSize=368 B 2023-04-09 07:54:51,779 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409786d7ee4e1534a8c892d57f22ae03589 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409786d7ee4e1534a8c892d57f22ae03589 2023-04-09 07:54:52,194 INFO [Listener at localhost/35977] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp/f1/7c76dd8da6ae4f668d06eb9872640ed4 2023-04-09 07:54:52,200 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp/f1/7c76dd8da6ae4f668d06eb9872640ed4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4 2023-04-09 07:54:52,204 INFO [Listener at localhost/35977] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4, entries=1, sequenceid=5, filesize=4.9 K 2023-04-09 07:54:52,205 INFO [Listener at localhost/35977] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 438a987d05d3cb50118ccd8c8e13e0c8 in 469ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:52,205 DEBUG [Listener at localhost/35977] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-04-09 07:54:52,206 DEBUG [Listener at localhost/35977] regionserver.HRegion(2446): Flush status journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:52,208 INFO [Listener at localhost/35977] regionserver.HRegion(2745): Flushing 438a987d05d3cb50118ccd8c8e13e0c8 1/1 column families, dataSize=37 B heapSize=368 B 2023-04-09 07:54:52,626 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304093cd1fd00e143414390b5553104e4aca8 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304093cd1fd00e143414390b5553104e4aca8 2023-04-09 07:54:53,032 INFO [Listener at localhost/35977] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp/f1/5f11b9e101224d3aab3e39bcfae3c18a 2023-04-09 07:54:53,037 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp/f1/5f11b9e101224d3aab3e39bcfae3c18a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a 2023-04-09 07:54:53,041 INFO [Listener at localhost/35977] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a, entries=1, sequenceid=9, filesize=4.9 K 2023-04-09 07:54:53,042 INFO [Listener at localhost/35977] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 438a987d05d3cb50118ccd8c8e13e0c8 in 834ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:53,043 DEBUG [Listener at localhost/35977] regionserver.HRegion(2446): Flush status journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:53,056 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-04-09 07:54:53,057 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-09 07:54:53,058 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-09 07:54:53,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:53,059 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027493059 2023-04-09 07:54:53,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:54:53,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-04-09 07:54:53,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-04-09 07:54:53,061 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x05ab9e98 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:54:53,066 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18085160, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:54:53,066 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230409 for table='testGetAfterCompaction' and column='f1' 2023-04-09 07:54:53,108 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,108 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,159 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,159 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,210 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,210 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,261 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,261 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,312 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,312 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,363 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,363 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,414 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,414 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,465 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,465 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,516 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,516 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,567 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,567 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,618 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,618 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,669 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,669 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,720 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,720 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,771 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,771 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,822 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,822 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,873 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,873 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,896 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409205ae3a48d814d889576b91cb520914e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409205ae3a48d814d889576b91cb520914e 2023-04-09 07:54:53,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testGetAfterCompaction' column='f1' 2023-04-09 07:54:53,915 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:54:53,917 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59664, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:54:53,924 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,924 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,926 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:54:53,928 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58344, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:54:53,948 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/f1/425d9dd9145b4c9b946c615fcf8d4f90 first=Optional[r1] last=Optional[r2] 2023-04-09 07:54:53,949 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{f1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/f1/425d9dd9145b4c9b946c615fcf8d4f90}] 2023-04-09 07:54:53,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/f1/425d9dd9145b4c9b946c615fcf8d4f90 for inclusion in 438a987d05d3cb50118ccd8c8e13e0c8/f1 2023-04-09 07:54:53,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-04-09 07:54:53,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:54:53,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:53,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/f1/425d9dd9145b4c9b946c615fcf8d4f90 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testGetAfterCompaction__ve5cl3tdcs4l1dv793jnj8qocdvc79aoa4hlq14lannlfaek64b8m5tvbv030fel/f1/425d9dd9145b4c9b946c615fcf8d4f90 2023-04-09 07:54:53,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testGetAfterCompaction__ve5cl3tdcs4l1dv793jnj8qocdvc79aoa4hlq14lannlfaek64b8m5tvbv030fel/f1/425d9dd9145b4c9b946c615fcf8d4f90 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_ 2023-04-09 07:54:53,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testGetAfterCompaction__ve5cl3tdcs4l1dv793jnj8qocdvc79aoa4hlq14lannlfaek64b8m5tvbv030fel/f1/425d9dd9145b4c9b946c615fcf8d4f90 into 438a987d05d3cb50118ccd8c8e13e0c8/f1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_ - updating store file list. 2023-04-09 07:54:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_ into 438a987d05d3cb50118ccd8c8e13e0c8/f1 2023-04-09 07:54:53,971 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testGetAfterCompaction__ve5cl3tdcs4l1dv793jnj8qocdvc79aoa4hlq14lannlfaek64b8m5tvbv030fel/f1/425d9dd9145b4c9b946c615fcf8d4f90 into 438a987d05d3cb50118ccd8c8e13e0c8/f1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_) 2023-04-09 07:54:53,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testGetAfterCompaction__ve5cl3tdcs4l1dv793jnj8qocdvc79aoa4hlq14lannlfaek64b8m5tvbv030fel/f1/425d9dd9145b4c9b946c615fcf8d4f90 2023-04-09 07:54:53,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): 438a987d05d3cb50118ccd8c8e13e0c8/f1 is initiating minor compaction 2023-04-09 07:54:53,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5e15876; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:53,972 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 438a987d05d3cb50118ccd8c8e13e0c8/f1 in testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:53,973 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp, totalSize=10.0 K 2023-04-09 07:54:53,973 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 5f11b9e101224d3aab3e39bcfae3c18a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:54:53,973 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 66c744fcd03a4a209691613cc9805a9f_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:54:53,975 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testGetAfterCompaction' column='f1' 2023-04-09 07:54:53,975 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,975 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:53,975 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:53,976 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:53,978 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304093cd1fd00e143414390b5553104e4aca8 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202304093cd1fd00e143414390b5553104e4aca8 2023-04-09 07:54:53,980 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409786d7ee4e1534a8c892d57f22ae03589 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409786d7ee4e1534a8c892d57f22ae03589 2023-04-09 07:54:53,980 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-04-09 07:54:53,980 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230409' 2023-04-09 07:54:53,983 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230409 for table='testGetAfterCompaction' and column='f1' 2023-04-09 07:54:53,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-04-09 07:54:53,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:54:53,984 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-04-09 07:54:53,986 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 438a987d05d3cb50118ccd8c8e13e0c8#f1#compaction#76 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:54:53,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 928 msec 2023-04-09 07:54:53,988 DEBUG [Listener at localhost/35977] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-09 07:54:53,997 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-09 07:54:53,998 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34907] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:53,998 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-04-09 07:54:53,998 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-09 07:54:53,998 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34907] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-09 07:54:53,998 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34907] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. because compaction request was cancelled 2023-04-09 07:54:54,050 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,050 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,050 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,101 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,101 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,101 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,152 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,152 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,152 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,203 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,203 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,203 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,254 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,254 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,254 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,305 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,305 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,305 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,356 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,356 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,357 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,407 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,407 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,407 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,458 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,458 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,458 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,509 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,509 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,510 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,560 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,561 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,561 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,611 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,612 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,612 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,662 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,663 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,663 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,669 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:54:54,714 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,714 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,714 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,765 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,765 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,765 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,810 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/.tmp/f1/1e7be946c3474b46827d6f4b988f7f7c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/1e7be946c3474b46827d6f4b988f7f7c 2023-04-09 07:54:54,816 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 438a987d05d3cb50118ccd8c8e13e0c8/f1 of 438a987d05d3cb50118ccd8c8e13e0c8 into 1e7be946c3474b46827d6f4b988f7f7c(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:54:54,816 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:54,816 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/1e7be946c3474b46827d6f4b988f7f7c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1681026894804; access_time=1681026893979; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,816 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1681026893972; duration=0sec 2023-04-09 07:54:54,816 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026893032; access_time=1681026892211; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,816 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1681026893891; access_time=1681026893080; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-04-09 07:54:54,816 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,817 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:54:54,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_] to archive 2023-04-09 07:54:54,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:54:54,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/5f11b9e101224d3aab3e39bcfae3c18a 2023-04-09 07:54:54,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/66c744fcd03a4a209691613cc9805a9f_SeqId_12_ 2023-04-09 07:54:54,867 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/1e7be946c3474b46827d6f4b988f7f7c; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1681026894804; access_time=1681026893979; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,867 INFO [Listener at localhost/35977] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1681026892194; access_time=1681026891744; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-04-09 07:54:54,988 INFO [Listener at localhost/35977] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-04-09 07:54:54,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-04-09 07:54:54,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,001 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026895001"}]},"ts":"1681026895001"} 2023-04-09 07:54:55,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-09 07:54:55,003 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-04-09 07:54:55,005 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-04-09 07:54:55,007 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, UNASSIGN}] 2023-04-09 07:54:55,008 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, UNASSIGN 2023-04-09 07:54:55,009 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=438a987d05d3cb50118ccd8c8e13e0c8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:55,009 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1681026895009"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026895009"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026895009"}]},"ts":"1681026895009"} 2023-04-09 07:54:55,010 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 438a987d05d3cb50118ccd8c8e13e0c8, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:55,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-09 07:54:55,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:55,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 438a987d05d3cb50118ccd8c8e13e0c8, disabling compactions & flushes 2023-04-09 07:54:55,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:55,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:55,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. after waiting 0 ms 2023-04-09 07:54:55,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:55,162 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-09 07:54:55,162 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-04-09 07:54:55,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-04-09 07:54:55,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-09 07:54:55,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8. 2023-04-09 07:54:55,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 438a987d05d3cb50118ccd8c8e13e0c8: 2023-04-09 07:54:55,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:55,171 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=438a987d05d3cb50118ccd8c8e13e0c8, regionState=CLOSED 2023-04-09 07:54:55,171 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1681026895171"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026895171"}]},"ts":"1681026895171"} 2023-04-09 07:54:55,175 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-04-09 07:54:55,175 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 438a987d05d3cb50118ccd8c8e13e0c8, server=jenkins-hbase4.apache.org,34907,1681026816245 in 163 msec 2023-04-09 07:54:55,177 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-04-09 07:54:55,177 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=438a987d05d3cb50118ccd8c8e13e0c8, UNASSIGN in 168 msec 2023-04-09 07:54:55,178 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026895178"}]},"ts":"1681026895178"} 2023-04-09 07:54:55,179 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-04-09 07:54:55,181 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-04-09 07:54:55,182 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 187 msec 2023-04-09 07:54:55,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-04-09 07:54:55,305 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-04-09 07:54:55,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-04-09 07:54:55,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,318 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,320 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-04-09 07:54:55,323 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:55,325 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1, FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/recovered.edits] 2023-04-09 07:54:55,332 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/1e7be946c3474b46827d6f4b988f7f7c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/1e7be946c3474b46827d6f4b988f7f7c 2023-04-09 07:54:55,334 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/f1/7c76dd8da6ae4f668d06eb9872640ed4 2023-04-09 07:54:55,337 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/recovered.edits/15.seqid to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8/recovered.edits/15.seqid 2023-04-09 07:54:55,338 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testGetAfterCompaction/438a987d05d3cb50118ccd8c8e13e0c8 2023-04-09 07:54:55,338 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-04-09 07:54:55,338 DEBUG [PEWorker-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-04-09 07:54:55,339 DEBUG [PEWorker-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-04-09 07:54:55,345 DEBUG [PEWorker-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409205ae3a48d814d889576b91cb520914e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230409205ae3a48d814d889576b91cb520914e 2023-04-09 07:54:55,345 DEBUG [PEWorker-3] backup.HFileArchiver(596): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-04-09 07:54:55,348 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,386 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-04-09 07:54:55,389 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-04-09 07:54:55,390 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,390 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-04-09 07:54:55,390 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1681026895390"}]},"ts":"9223372036854775807"} 2023-04-09 07:54:55,392 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-09 07:54:55,392 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 438a987d05d3cb50118ccd8c8e13e0c8, NAME => 'testGetAfterCompaction,,1681026891103.438a987d05d3cb50118ccd8c8e13e0c8.', STARTKEY => '', ENDKEY => ''}] 2023-04-09 07:54:55,392 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-04-09 07:54:55,392 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1681026895392"}]},"ts":"9223372036854775807"} 2023-04-09 07:54:55,395 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-04-09 07:54:55,399 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-04-09 07:54:55,401 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 88 msec 2023-04-09 07:54:55,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-04-09 07:54:55,422 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-04-09 07:54:55,439 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=260 (was 256) - Thread LEAK? -, OpenFileDescriptor=656 (was 650) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 294) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1628 (was 1635) 2023-04-09 07:54:55,453 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=260, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=169, AvailableMemoryMB=1627 2023-04-09 07:54:55,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-09 07:54:55,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-04-09 07:54:55,458 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:54:55,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-04-09 07:54:55,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:55,462 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:54:55,480 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:55,481 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d empty. 2023-04-09 07:54:55,481 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:55,481 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:55,483 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee empty. 2023-04-09 07:54:55,483 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:55,488 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:55,488 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a empty. 2023-04-09 07:54:55,489 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:55,489 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-04-09 07:54:55,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:55,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:55,908 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-04-09 07:54:55,910 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => d4ff53eca0e32f1940dd88a1cd07958a, NAME => 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:55,910 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => 9be6d4060ae57ce72cac4673686ad32d, NAME => 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:55,910 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => 60a1e7a3b2b7470f369b184b87cd1dee, NAME => 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:54:55,930 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:55,930 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing 9be6d4060ae57ce72cac4673686ad32d, disabling compactions & flushes 2023-04-09 07:54:55,930 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:55,931 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:55,931 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. after waiting 0 ms 2023-04-09 07:54:55,931 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:55,931 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:55,931 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:54:56,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing 60a1e7a3b2b7470f369b184b87cd1dee, disabling compactions & flushes 2023-04-09 07:54:56,332 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. after waiting 0 ms 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,332 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,332 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:54:56,346 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:56,347 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing d4ff53eca0e32f1940dd88a1cd07958a, disabling compactions & flushes 2023-04-09 07:54:56,347 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,347 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,347 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. after waiting 0 ms 2023-04-09 07:54:56,347 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,347 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,347 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:54:56,349 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:54:56,350 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026896350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026896350"}]},"ts":"1681026896350"} 2023-04-09 07:54:56,350 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026896350"}]},"ts":"1681026896350"} 2023-04-09 07:54:56,350 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026896350"}]},"ts":"1681026896350"} 2023-04-09 07:54:56,352 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-09 07:54:56,353 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:54:56,353 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026896353"}]},"ts":"1681026896353"} 2023-04-09 07:54:56,354 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-04-09 07:54:56,360 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN}] 2023-04-09 07:54:56,361 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN 2023-04-09 07:54:56,361 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN 2023-04-09 07:54:56,362 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN 2023-04-09 07:54:56,364 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:56,364 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:56,364 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:54:56,516 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,516 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,516 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896515"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026896515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026896515"}]},"ts":"1681026896515"} 2023-04-09 07:54:56,516 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,516 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026896515"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026896515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026896515"}]},"ts":"1681026896515"} 2023-04-09 07:54:56,516 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896515"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026896515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026896515"}]},"ts":"1681026896515"} 2023-04-09 07:54:56,517 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:56,523 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:56,525 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:54:56,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:56,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:56,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9be6d4060ae57ce72cac4673686ad32d, NAME => 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.', STARTKEY => '1', ENDKEY => '2'} 2023-04-09 07:54:56,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:56,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,680 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,681 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:54:56,682 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:54:56,682 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9be6d4060ae57ce72cac4673686ad32d columnFamilyName family1 2023-04-09 07:54:56,683 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(310): Store=9be6d4060ae57ce72cac4673686ad32d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,684 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,685 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2 2023-04-09 07:54:56,685 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2 2023-04-09 07:54:56,685 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9be6d4060ae57ce72cac4673686ad32d columnFamilyName family2 2023-04-09 07:54:56,687 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(310): Store=9be6d4060ae57ce72cac4673686ad32d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:56,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:54:56,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:56,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9be6d4060ae57ce72cac4673686ad32d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62559219, jitterRate=-0.06779499351978302}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:56,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:54:56,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., pid=84, masterSystemTime=1681026896675 2023-04-09 07:54:56,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:56,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:56,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 60a1e7a3b2b7470f369b184b87cd1dee, NAME => 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.', STARTKEY => '2', ENDKEY => ''} 2023-04-09 07:54:56,699 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,699 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026896699"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026896699"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026896699"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026896699"}]},"ts":"1681026896699"} 2023-04-09 07:54:56,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:56,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,701 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-04-09 07:54:56,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245 in 179 msec 2023-04-09 07:54:56,703 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:54:56,703 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:54:56,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN in 344 msec 2023-04-09 07:54:56,704 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a1e7a3b2b7470f369b184b87cd1dee columnFamilyName family1 2023-04-09 07:54:56,704 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(310): Store=60a1e7a3b2b7470f369b184b87cd1dee/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,705 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,719 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2 2023-04-09 07:54:56,719 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2 2023-04-09 07:54:56,719 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a1e7a3b2b7470f369b184b87cd1dee columnFamilyName family2 2023-04-09 07:54:56,720 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(310): Store=60a1e7a3b2b7470f369b184b87cd1dee/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:56,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:54:56,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:56,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 60a1e7a3b2b7470f369b184b87cd1dee; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61284518, jitterRate=-0.08678951859474182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:56,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:54:56,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., pid=83, masterSystemTime=1681026896675 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:56,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4ff53eca0e32f1940dd88a1cd07958a, NAME => 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.', STARTKEY => '', ENDKEY => '1'} 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,732 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,732 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896732"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026896732"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026896732"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026896732"}]},"ts":"1681026896732"} 2023-04-09 07:54:56,735 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-04-09 07:54:56,736 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245 in 216 msec 2023-04-09 07:54:56,737 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN in 376 msec 2023-04-09 07:54:56,751 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,753 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:54:56,753 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:54:56,754 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4ff53eca0e32f1940dd88a1cd07958a columnFamilyName family1 2023-04-09 07:54:56,755 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(310): Store=d4ff53eca0e32f1940dd88a1cd07958a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,755 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,757 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2 2023-04-09 07:54:56,757 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2 2023-04-09 07:54:56,757 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4ff53eca0e32f1940dd88a1cd07958a columnFamilyName family2 2023-04-09 07:54:56,758 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(310): Store=d4ff53eca0e32f1940dd88a1cd07958a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:54:56,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:54:56,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:54:56,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:54:56,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d4ff53eca0e32f1940dd88a1cd07958a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66429832, jitterRate=-0.010118365287780762}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:54:56,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:54:56,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., pid=85, masterSystemTime=1681026896675 2023-04-09 07:54:56,770 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:56,770 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026896770"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026896770"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026896770"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026896770"}]},"ts":"1681026896770"} 2023-04-09 07:54:56,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:56,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-04-09 07:54:56,773 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245 in 250 msec 2023-04-09 07:54:56,775 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-04-09 07:54:56,775 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN in 415 msec 2023-04-09 07:54:56,775 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:54:56,775 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026896775"}]},"ts":"1681026896775"} 2023-04-09 07:54:56,776 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-04-09 07:54:56,778 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:54:56,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.3220 sec 2023-04-09 07:54:57,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-09 07:54:57,566 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-04-09 07:54:57,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:57,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:57,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:57,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:57,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-09 07:54:57,588 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-09 07:54:57,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:57,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-09 07:54:57,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-09 07:54:57,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:57,588 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:57,590 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:57,590 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:57,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:57,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:57,590 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:57,590 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:57,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:57,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:57,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:57,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:57,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-09 07:54:57,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-09 07:54:57,591 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-09 07:54:57,591 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:57,591 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-09 07:54:57,591 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:57,591 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:54:57,592 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:57,592 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9be6d4060ae57ce72cac4673686ad32d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:54:57,593 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:57,592 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:54:57,593 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:54:57,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:54:57,593 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:57,593 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 60a1e7a3b2b7470f369b184b87cd1dee 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:54:57,595 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d4ff53eca0e32f1940dd88a1cd07958a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:54:57,638 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 2023-04-09 07:54:57,648 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/6e82ea65bd89416ba123d8deb4c583f2 2023-04-09 07:54:58,026 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d 2023-04-09 07:54:58,036 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 2023-04-09 07:54:58,064 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94 2023-04-09 07:54:58,431 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/dc3980414a4348f6a3429b4c34391e56 2023-04-09 07:54:58,441 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/ab76dd4699904d23a90e5fb92ed66388 2023-04-09 07:54:58,469 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/32d1220d842340d789e60950b9a1ba3e 2023-04-09 07:54:58,474 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/6e82ea65bd89416ba123d8deb4c583f2 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2 2023-04-09 07:54:58,477 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:54:58,478 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/32d1220d842340d789e60950b9a1ba3e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e 2023-04-09 07:54:58,482 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:54:58,483 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 60a1e7a3b2b7470f369b184b87cd1dee in 890ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:58,483 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-04-09 07:54:58,484 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:54:58,484 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:58,484 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:54:58,849 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f 2023-04-09 07:54:58,859 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/048a51cc9c4e4092893240c4631a54e3 2023-04-09 07:54:58,863 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4 2023-04-09 07:54:58,867 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/dc3980414a4348f6a3429b4c34391e56 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56 2023-04-09 07:54:58,871 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:54:58,871 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/670de79c0d7147e29301d37b7d05be6a 2023-04-09 07:54:58,872 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/048a51cc9c4e4092893240c4631a54e3 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3 2023-04-09 07:54:58,877 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/ab76dd4699904d23a90e5fb92ed66388 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388 2023-04-09 07:54:58,878 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:54:58,879 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9be6d4060ae57ce72cac4673686ad32d in 1287ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:58,880 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:54:58,880 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:58,880 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:54:58,881 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:54:58,882 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/670de79c0d7147e29301d37b7d05be6a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a 2023-04-09 07:54:58,886 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:54:58,887 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d4ff53eca0e32f1940dd88a1cd07958a in 1292ms, sequenceid=5, compaction requested=false 2023-04-09 07:54:58,887 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:54:58,887 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-09 07:54:58,887 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-09 07:54:58,889 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,889 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:58,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:58,890 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:58,890 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:54:58,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:58,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:58,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-09 07:54:58,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@553caf30[Count = 0] remaining members to acquire global barrier 2023-04-09 07:54:58,891 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-09 07:54:58,891 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,894 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,894 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-09 07:54:58,894 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-09 07:54:58,894 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:54:58,894 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,894 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:54:58,895 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-09 07:54:58,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:58,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:58,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:58,895 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-09 07:54:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:58,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-09 07:54:58,898 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-09 07:54:58,898 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:54:58,898 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:54:58,898 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-09 07:54:58,898 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:54:58,900 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,900 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,900 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:58,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:54:58,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:58,900 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:54:58,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,902 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:54:58,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:54:58,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,904 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:58,904 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:54:58,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:58,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:58,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-09 07:54:58,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:54:58,905 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:58,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:54:58,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-09 07:54:58,905 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:54:59,006 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:54:59,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:54:59,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:59,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:59,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:54:59,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:54:59,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-09 07:54:59,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-09 07:54:59,013 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-09 07:54:59,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:59,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-09 07:54:59,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-09 07:54:59,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:59,014 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:59,016 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:54:59,016 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:59,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:59,016 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:54:59,016 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:54:59,017 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:54:59,017 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-09 07:54:59,017 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-09 07:54:59,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-09 07:54:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-09 07:54:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:54:59,018 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:54:59,032 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:54:59,032 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:59,032 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9be6d4060ae57ce72cac4673686ad32d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:54:59,032 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 60a1e7a3b2b7470f369b184b87cd1dee 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:54:59,032 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:54:59,033 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d4ff53eca0e32f1940dd88a1cd07958a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:54:59,088 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f 2023-04-09 07:54:59,089 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 2023-04-09 07:54:59,489 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 2023-04-09 07:54:59,498 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/5987d3843a9541ec9705d9216fa5d916 2023-04-09 07:54:59,506 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/d251bda3f1d84e7f94675a658f2caf7e 2023-04-09 07:54:59,511 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/c2fd7100e7464d1bbfa901c01d8cf756 2023-04-09 07:54:59,529 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25 2023-04-09 07:54:59,545 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/c60cda638dab443ba1daf428f043a481 2023-04-09 07:54:59,549 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/c2fd7100e7464d1bbfa901c01d8cf756 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756 2023-04-09 07:54:59,553 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:54:59,554 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/c60cda638dab443ba1daf428f043a481 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481 2023-04-09 07:54:59,557 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:54:59,558 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d4ff53eca0e32f1940dd88a1cd07958a in 525ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:59,558 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:54:59,558 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:54:59,558 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:54:59,915 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e 2023-04-09 07:54:59,921 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/5d4812eb929743ffb69372e49be65034 2023-04-09 07:54:59,925 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4 2023-04-09 07:54:59,926 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/5987d3843a9541ec9705d9216fa5d916 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916 2023-04-09 07:54:59,931 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:54:59,932 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/5d4812eb929743ffb69372e49be65034 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034 2023-04-09 07:54:59,936 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:54:59,937 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9be6d4060ae57ce72cac4673686ad32d in 905ms, sequenceid=9, compaction requested=false 2023-04-09 07:54:59,937 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:54:59,937 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:54:59,937 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:00,331 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/c797a98e70fa4ba1988dbd4005f84ea9 2023-04-09 07:55:00,337 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/d251bda3f1d84e7f94675a658f2caf7e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e 2023-04-09 07:55:00,341 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:55:00,342 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/c797a98e70fa4ba1988dbd4005f84ea9 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9 2023-04-09 07:55:00,347 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:00,348 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 60a1e7a3b2b7470f369b184b87cd1dee in 1316ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:00,348 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:00,348 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:00,348 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:00,348 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:00,348 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,348 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-09 07:55:00,348 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-09 07:55:00,351 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:00,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:00,351 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,352 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,352 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:00,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:00,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:00,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:00,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-09 07:55:00,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e994f98[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:00,354 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-09 07:55:00,354 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,355 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-09 07:55:00,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-09 07:55:00,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:00,355 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,355 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:00,357 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:00,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:00,357 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-09 07:55:00,357 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:00,357 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-09 07:55:00,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:00,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:00,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:00,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:00,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-09 07:55:00,361 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-09 07:55:00,361 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:00,361 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:00,361 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-09 07:55:00,361 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:00,362 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,363 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:00,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,363 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:00,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:00,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:00,363 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:00,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,365 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:00,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:00,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,401 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0df9081b to 127.0.0.1:57867 2023-04-09 07:55:00,401 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,401 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:00,401 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x05ab9e98 to 127.0.0.1:57867 2023-04-09 07:55:00,401 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,402 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:00,402 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b6cade2 to 127.0.0.1:57867 2023-04-09 07:55:00,402 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,402 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x342c8467 to 127.0.0.1:57867 2023-04-09 07:55:00,402 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,402 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:00,402 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6c048f43 to 127.0.0.1:57867 2023-04-09 07:55:00,402 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,402 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:00,402 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x423f78b7 to 127.0.0.1:57867 2023-04-09 07:55:00,402 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,403 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:00,403 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x34df3cb3 to 127.0.0.1:57867 2023-04-09 07:55:00,403 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:00,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:00,404 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:00,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:00,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-09 07:55:00,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:00,404 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,405 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:00,405 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:00,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:00,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:00,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-04-09 07:55:00,505 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:00,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:00,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:00,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:00,508 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:00,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:00,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:00,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-09 07:55:00,513 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-09 07:55:00,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:00,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-09 07:55:00,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-09 07:55:00,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,513 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:00,515 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:00,515 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:00,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:00,515 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:00,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:00,515 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:00,516 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:00,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:00,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-09 07:55:00,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-09 07:55:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-09 07:55:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:00,517 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:00,520 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9be6d4060ae57ce72cac4673686ad32d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:00,520 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:00,520 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:55:00,520 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:55:00,520 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d4ff53eca0e32f1940dd88a1cd07958a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:00,520 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 60a1e7a3b2b7470f369b184b87cd1dee 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:00,561 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e 2023-04-09 07:55:00,566 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/489073fa9db54eae8fe5585df2e0d914 2023-04-09 07:55:00,670 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:00,962 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 2023-04-09 07:55:00,964 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 2023-04-09 07:55:00,978 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec 2023-04-09 07:55:01,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/0512b4adaf1b4014a3bf577864b0cd49 2023-04-09 07:55:01,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/d33553a5e93f408eacec7b222611058a 2023-04-09 07:55:01,383 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/338c2bebb8d54f2880050184268fa31e 2023-04-09 07:55:01,389 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/489073fa9db54eae8fe5585df2e0d914 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914 2023-04-09 07:55:01,394 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:01,395 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/338c2bebb8d54f2880050184268fa31e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e 2023-04-09 07:55:01,400 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:01,401 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 60a1e7a3b2b7470f369b184b87cd1dee in 880ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:01,401 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:01,401 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:01,401 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:01,413 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284 2023-04-09 07:55:01,817 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca 2023-04-09 07:55:01,827 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/e0a6f1d523c140308259de663aae00e6 2023-04-09 07:55:01,831 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/0512b4adaf1b4014a3bf577864b0cd49 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49 2023-04-09 07:55:01,835 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:01,836 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/e0a6f1d523c140308259de663aae00e6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6 2023-04-09 07:55:01,840 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:01,841 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9be6d4060ae57ce72cac4673686ad32d in 1321ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:01,841 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:01,841 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:01,841 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:02,222 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/4f2bfb0317dc40c397208cd03f0e4579 2023-04-09 07:55:02,227 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/d33553a5e93f408eacec7b222611058a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a 2023-04-09 07:55:02,243 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:02,245 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/4f2bfb0317dc40c397208cd03f0e4579 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579 2023-04-09 07:55:02,256 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:02,257 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d4ff53eca0e32f1940dd88a1cd07958a in 1737ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:02,257 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:02,257 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:02,257 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:02,257 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:02,257 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,257 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-09 07:55:02,257 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-09 07:55:02,259 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,259 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:02,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:02,259 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,259 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:02,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:02,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:02,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:02,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-09 07:55:02,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79fe4fc1[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:02,261 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-09 07:55:02,262 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,263 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,263 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-09 07:55:02,263 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-09 07:55:02,263 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:02,263 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,263 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:02,271 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-09 07:55:02,271 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,271 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:02,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:02,271 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-09 07:55:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:02,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:02,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-09 07:55:02,274 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-09 07:55:02,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:02,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:02,274 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-09 07:55:02,274 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:02,276 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,276 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,276 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:02,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:02,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:02,276 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:02,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,278 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:02,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:02,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:02,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:02,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:02,281 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:02,280 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:02,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-09 07:55:02,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:02,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:02,281 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:02,381 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:02,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:02,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:02,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:02,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:02,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:02,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:02,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-09 07:55:02,390 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-09 07:55:02,390 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:02,390 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-09 07:55:02,390 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-09 07:55:02,391 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,391 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:02,392 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:02,392 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:02,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:02,393 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:02,393 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:02,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:02,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:02,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:02,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-09 07:55:02,395 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-09 07:55:02,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-09 07:55:02,395 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-09 07:55:02,395 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:02,395 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-09 07:55:02,395 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:02,395 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:55:02,395 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9be6d4060ae57ce72cac4673686ad32d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:02,396 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:02,396 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:55:02,396 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:02,396 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:02,396 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:02,396 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 60a1e7a3b2b7470f369b184b87cd1dee 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:02,396 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:55:02,400 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d4ff53eca0e32f1940dd88a1cd07958a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:02,482 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f 2023-04-09 07:55:02,482 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 2023-04-09 07:55:02,882 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 2023-04-09 07:55:02,896 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/8530d077b2fb44e8b1bf6dbdc609356d 2023-04-09 07:55:02,911 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/c4a5a528cb424261a35132ab5f18fcd0 2023-04-09 07:55:02,944 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290 2023-04-09 07:55:02,962 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 2023-04-09 07:55:02,967 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/8530d077b2fb44e8b1bf6dbdc609356d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d 2023-04-09 07:55:02,974 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:02,976 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 2023-04-09 07:55:02,980 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:02,981 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9be6d4060ae57ce72cac4673686ad32d in 586ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:02,982 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:02,982 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:02,982 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:03,302 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/30ab0249d92e4fdcb426d4272ddba3be 2023-04-09 07:55:03,318 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440 2023-04-09 07:55:03,343 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce 2023-04-09 07:55:03,725 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/24465f8bedd54e74b826892b610dcd3c 2023-04-09 07:55:03,730 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/30ab0249d92e4fdcb426d4272ddba3be as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be 2023-04-09 07:55:03,734 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:03,734 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/24465f8bedd54e74b826892b610dcd3c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c 2023-04-09 07:55:03,739 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:03,740 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d4ff53eca0e32f1940dd88a1cd07958a in 1343ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:03,740 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:03,740 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:03,740 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:03,749 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/9050132042ca4d8296073af7e0057ac2 2023-04-09 07:55:03,753 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/c4a5a528cb424261a35132ab5f18fcd0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0 2023-04-09 07:55:03,757 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:03,758 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/9050132042ca4d8296073af7e0057ac2 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2 2023-04-09 07:55:03,770 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:03,771 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 60a1e7a3b2b7470f369b184b87cd1dee in 1375ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:03,771 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:03,771 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-09 07:55:03,771 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-09 07:55:03,773 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,773 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:03,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:03,774 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,774 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:03,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:03,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:03,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:03,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-09 07:55:03,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ea0a3b4[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:03,776 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-09 07:55:03,776 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,777 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,778 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:03,780 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,780 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-09 07:55:03,780 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-09 07:55:03,780 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:03,782 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,782 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-09 07:55:03,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:03,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:03,782 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:03,782 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-09 07:55:03,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:03,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:03,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:03,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:03,785 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-09 07:55:03,786 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:03,786 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:03,786 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-09 07:55:03,786 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:03,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-09 07:55:03,787 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,787 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:03,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,788 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:03,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,789 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,790 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:03,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,793 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-09 07:55:03,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:03,794 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:03,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:03,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:03,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:03,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:03,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:03,894 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:03,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:03,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:03,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1681026903907 (current time:1681026903907). 2023-04-09 07:55:03,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-04-09 07:55:03,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] zookeeper.ReadOnlyZKClient(139): Connect 0x4b948d1a to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:03,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ba8c6e5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:03,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:03,924 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46826, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b948d1a to 127.0.0.1:57867 2023-04-09 07:55:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:03,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-04-09 07:55:03,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:03,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:03,935 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:03,936 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1681027503936 2023-04-09 07:55:03,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:03,944 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1681026903895 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-04-09 07:55:03,945 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:03,946 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 11 msec 2023-04-09 07:55:03,947 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-09 07:55:03,948 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-09 07:55:03,948 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1681027503948 2023-04-09 07:55:03,950 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:03,950 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-04-09 07:55:03,960 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1681026903895 2023-04-09 07:55:03,964 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1681026903895' 2023-04-09 07:55:03,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-04-09 07:55:03,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1681026903895' starting 'acquire' 2023-04-09 07:55:03,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1681026903895', kicking off acquire phase on members. 2023-04-09 07:55:03,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:03,965 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1681026903895 2023-04-09 07:55:03,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-04-09 07:55:03,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-04-09 07:55:03,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-09 07:55:03,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:03,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1681026903895 2023-04-09 07:55:03,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:03,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-04-09 07:55:03,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1681026903895 2023-04-09 07:55:03,969 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1681026903895 from table testCompactionWithHFileLink type FLUSH 2023-04-09 07:55:03,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1681026903895 2023-04-09 07:55:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1681026903895' with timeout 300000ms 2023-04-09 07:55:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-04-09 07:55:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1681026903895' starting 'acquire' stage 2023-04-09 07:55:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1681026903895' locally acquired 2023-04-09 07:55:03,972 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (snaptb-1681026903895) in zk 2023-04-09 07:55:03,973 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-09 07:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:03,974 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1681026903895' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:03,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:03,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'snaptb-1681026903895' on coordinator 2023-04-09 07:55:03,976 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ed5ca7f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:03,976 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1681026903895' starting 'in-barrier' execution. 2023-04-09 07:55:03,976 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,977 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:03,978 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1681026903895' received 'reached' from coordinator. 2023-04-09 07:55:03,978 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:03,978 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:03,980 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.' region-info for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,980 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.' region-info for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-04-09 07:55:03,984 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:03,985 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:55:03,985 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:03,985 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.' region-info for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-04-09 07:55:03,987 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-09 07:55:03,987 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-09 07:55:03,987 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d] hfiles 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be] hfiles 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0] hfiles 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,991 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,995 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2] hfiles 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7] hfiles 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c] hfiles 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2 for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c for snapshot=snaptb-1681026903895 2023-04-09 07:55:03,996 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 for snapshot=snaptb-1681026903895 2023-04-09 07:55:04,022 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. completed. 2023-04-09 07:55:04,022 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:04,023 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-04-09 07:55:04,050 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:04,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:04,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:04,055 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-04-09 07:55:04,255 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:04,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:04,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:04,257 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-04-09 07:55:04,430 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. completed. 2023-04-09 07:55:04,430 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:04,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-04-09 07:55:04,439 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. completed. 2023-04-09 07:55:04,439 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:04,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-04-09 07:55:04,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-04-09 07:55:04,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1681026903895' locally completed 2023-04-09 07:55:04,439 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1681026903895' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:04,441 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,441 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1681026903895' has notified controller of completion 2023-04-09 07:55:04,441 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-09 07:55:04,441 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1681026903895' completed. 2023-04-09 07:55:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:04,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:04,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:04,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:04,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:04,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1681026903895' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:04,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'snaptb-1681026903895', counting down latch. Waiting for 0 more 2023-04-09 07:55:04,444 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1681026903895' execution completed 2023-04-09 07:55:04,444 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:04,444 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:04,444 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1681026903895 2023-04-09 07:55:04,444 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1681026903895including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-04-09 07:55:04,446 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,446 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:04,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-04-09 07:55:04,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,446 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-04-09 07:55:04,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:04,446 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-04-09 07:55:04,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-09 07:55:04,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:04,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:04,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:04,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:04,449 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1681026903895 2023-04-09 07:55:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-04-09 07:55:04,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:04,453 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1681026903895/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:04,453 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1681026903895 2023-04-09 07:55:04,453 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-04-09 07:55:04,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-04-09 07:55:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504] mob files 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 2023-04-09 07:55:04,455 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 2023-04-09 07:55:04,456 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f 2023-04-09 07:55:04,456 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440] mob files 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94 2023-04-09 07:55:04,457 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25 2023-04-09 07:55:04,458 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca 2023-04-09 07:55:04,458 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4 2023-04-09 07:55:04,458 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440 2023-04-09 07:55:04,465 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1681026903895 2023-04-09 07:55:04,467 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/snaptb-1681026903895 2023-04-09 07:55:04,557 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:04,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:04,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:04,558 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-04-09 07:55:04,931 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/snaptb-1681026903895 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/snaptb-1681026903895 2023-04-09 07:55:05,016 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1681026903895 of table testCompactionWithHFileLink completed 2023-04-09 07:55:05,016 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/snaptb-1681026903895 2023-04-09 07:55:05,017 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/snaptb-1681026903895 2023-04-09 07:55:05,017 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1681026903895 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1681026903944Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1681026904465 (+521 ms)Consolidate snapshot: snaptb-1681026903895 at 1681026904465Loading Region manifests for snaptb-1681026903895 at 1681026904465Writing data manifest for snaptb-1681026903895 at 1681026904476 (+11 ms)Verifying snapshot: snaptb-1681026903895 at 1681026904893 (+417 ms)Snapshot snaptb-1681026903895 of table testCompactionWithHFileLink completed at 1681026905016 (+123 ms) 2023-04-09 07:55:05,018 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-04-09 07:55:05,019 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.0710 sec 2023-04-09 07:55:05,058 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:05,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:05,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-04-09 07:55:05,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:05,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:05,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-04-09 07:55:05,109 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-04-09 07:55:05,110 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:05,110 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-04-09 07:55:05,110 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-04-09 07:55:05,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:05,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:05,114 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:05,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:05,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:05,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:05,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:05,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:05,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:05,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:05,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:05,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:05,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-04-09 07:55:05,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-04-09 07:55:05,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-04-09 07:55:05,115 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-04-09 07:55:05,116 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:05,116 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-04-09 07:55:05,126 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:05,126 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. started... 2023-04-09 07:55:05,126 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:05,126 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. started... 2023-04-09 07:55:05,126 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9be6d4060ae57ce72cac4673686ad32d 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:05,126 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 60a1e7a3b2b7470f369b184b87cd1dee 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:05,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:05,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:05,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:05,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. started... 2023-04-09 07:55:05,128 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d4ff53eca0e32f1940dd88a1cd07958a 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:05,162 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-04-09 07:55:05,213 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/f1626be71f51415ba6c8e09d4ac42696 2023-04-09 07:55:05,217 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1626be71f51415ba6c8e09d4ac42696 2023-04-09 07:55:05,235 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/8dcf4c9304a34944b4f5b0b92163f115 2023-04-09 07:55:05,238 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8dcf4c9304a34944b4f5b0b92163f115 2023-04-09 07:55:05,239 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/f1626be71f51415ba6c8e09d4ac42696 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696 2023-04-09 07:55:05,243 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1626be71f51415ba6c8e09d4ac42696 2023-04-09 07:55:05,243 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696, entries=3, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:05,244 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/8dcf4c9304a34944b4f5b0b92163f115 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115 2023-04-09 07:55:05,248 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8dcf4c9304a34944b4f5b0b92163f115 2023-04-09 07:55:05,248 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115, entries=1, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:05,249 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 60a1e7a3b2b7470f369b184b87cd1dee in 123ms, sequenceid=24, compaction requested=true 2023-04-09 07:55:05,249 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:05,249 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:05,249 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:05,602 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/6276f06743e9450494a25502acdec924 2023-04-09 07:55:05,607 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6276f06743e9450494a25502acdec924 2023-04-09 07:55:05,610 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e 2023-04-09 07:55:05,615 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a16bd7aedc44b3f82e54ad7a5b7d60e 2023-04-09 07:55:05,624 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/66d6d28e0e524f488bc4bc5a9d649139 2023-04-09 07:55:05,627 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66d6d28e0e524f488bc4bc5a9d649139 2023-04-09 07:55:05,628 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/6276f06743e9450494a25502acdec924 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924 2023-04-09 07:55:05,632 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6276f06743e9450494a25502acdec924 2023-04-09 07:55:05,632 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924, entries=3, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:05,633 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/66d6d28e0e524f488bc4bc5a9d649139 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139 2023-04-09 07:55:05,636 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 66d6d28e0e524f488bc4bc5a9d649139 2023-04-09 07:55:05,637 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139, entries=1, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:05,637 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9be6d4060ae57ce72cac4673686ad32d in 511ms, sequenceid=24, compaction requested=true 2023-04-09 07:55:05,637 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:05,637 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:05,637 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:06,431 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/6b9e5087285047d0983509a11dad8655 2023-04-09 07:55:06,448 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b9e5087285047d0983509a11dad8655 2023-04-09 07:55:06,449 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e 2023-04-09 07:55:06,455 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a16bd7aedc44b3f82e54ad7a5b7d60e 2023-04-09 07:55:06,455 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e, entries=3, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:06,456 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/6b9e5087285047d0983509a11dad8655 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655 2023-04-09 07:55:06,461 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b9e5087285047d0983509a11dad8655 2023-04-09 07:55:06,461 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655, entries=1, sequenceid=24, filesize=5.0 K 2023-04-09 07:55:06,461 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d4ff53eca0e32f1940dd88a1cd07958a in 1333ms, sequenceid=24, compaction requested=true 2023-04-09 07:55:06,462 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:06,462 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:06,462 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:06,462 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:06,462 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,462 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-04-09 07:55:06,462 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-04-09 07:55:06,464 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,464 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:06,464 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,464 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:06,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:06,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:06,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:06,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-04-09 07:55:06,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a14a5b2[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:06,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-04-09 07:55:06,466 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,471 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-04-09 07:55:06,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-04-09 07:55:06,471 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:06,471 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,471 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:06,472 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,472 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-04-09 07:55:06,472 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:06,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:06,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:06,472 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-04-09 07:55:06,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:06,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:06,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:06,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:06,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-04-09 07:55:06,475 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-04-09 07:55:06,476 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:06,476 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:06,476 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-04-09 07:55:06,476 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:06,477 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,477 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:06,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,477 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:06,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:06,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:06,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:06,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:06,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,479 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:06,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,480 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:06,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-04-09 07:55:06,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-04-09 07:55:06,482 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:06,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:06,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:06,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-04-09 07:55:06,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:06,482 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,483 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-04-09 07:55:06,483 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-04-09 07:55:06,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:06,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:06,483 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:06,483 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:06,583 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:06,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:06,585 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:06,585 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 9be6d4060ae57ce72cac4673686ad32d/family1 is initiating major compaction (all files) 2023-04-09 07:55:06,585 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:06,585 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:06,586 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 9be6d4060ae57ce72cac4673686ad32d/family1 in testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:06,586 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp, totalSize=26.7 K 2023-04-09 07:55:06,587 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting dc3980414a4348f6a3429b4c34391e56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897581 2023-04-09 07:55:06,587 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 5987d3843a9541ec9705d9216fa5d916, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:06,588 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 0512b4adaf1b4014a3bf577864b0cd49, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:06,588 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 8530d077b2fb44e8b1bf6dbdc609356d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:06,588 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 6276f06743e9450494a25502acdec924, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:06,605 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 9be6d4060ae57ce72cac4673686ad32d#family1#compaction#107 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:07,017 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del 2023-04-09 07:55:07,017 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del 2023-04-09 07:55:07,830 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 350db1ee5e6f427b907a8ac75d7bcd87 2023-04-09 07:55:07,831 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/350db1ee5e6f427b907a8ac75d7bcd87 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/350db1ee5e6f427b907a8ac75d7bcd87 2023-04-09 07:55:07,834 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 350db1ee5e6f427b907a8ac75d7bcd87 2023-04-09 07:55:07,835 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9be6d4060ae57ce72cac4673686ad32d/family1 of 9be6d4060ae57ce72cac4673686ad32d into 350db1ee5e6f427b907a8ac75d7bcd87(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:07,835 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:07,835 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:07,835 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 9be6d4060ae57ce72cac4673686ad32d/family2 is initiating major compaction (all files) 2023-04-09 07:55:07,836 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:07,836 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:07,836 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 9be6d4060ae57ce72cac4673686ad32d/family2 in testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:07,836 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp, totalSize=25.4 K 2023-04-09 07:55:07,836 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 048a51cc9c4e4092893240c4631a54e3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897581 2023-04-09 07:55:07,837 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 5d4812eb929743ffb69372e49be65034, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:07,837 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting e0a6f1d523c140308259de663aae00e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:07,837 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting b33d9c23d2a24add9cd9e8dd3382c4a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:07,837 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 66d6d28e0e524f488bc4bc5a9d649139, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:07,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924] to archive 2023-04-09 07:55:07,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:07,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/dc3980414a4348f6a3429b4c34391e56 2023-04-09 07:55:07,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5987d3843a9541ec9705d9216fa5d916 2023-04-09 07:55:07,848 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 9be6d4060ae57ce72cac4673686ad32d#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:07,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/0512b4adaf1b4014a3bf577864b0cd49 2023-04-09 07:55:07,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/8530d077b2fb44e8b1bf6dbdc609356d 2023-04-09 07:55:07,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/6276f06743e9450494a25502acdec924 2023-04-09 07:55:08,261 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del 2023-04-09 07:55:08,261 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del 2023-04-09 07:55:09,074 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80998ca50f1246d5afea4df7caeb3443 2023-04-09 07:55:09,074 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/80998ca50f1246d5afea4df7caeb3443 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/80998ca50f1246d5afea4df7caeb3443 2023-04-09 07:55:09,077 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80998ca50f1246d5afea4df7caeb3443 2023-04-09 07:55:09,078 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9be6d4060ae57ce72cac4673686ad32d/family2 of 9be6d4060ae57ce72cac4673686ad32d into 80998ca50f1246d5afea4df7caeb3443(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:09,078 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:09,078 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:09,079 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 60a1e7a3b2b7470f369b184b87cd1dee/family1 is initiating major compaction (all files) 2023-04-09 07:55:09,079 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:09,079 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:09,079 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 60a1e7a3b2b7470f369b184b87cd1dee/family1 in testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:09,079 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp, totalSize=26.7 K 2023-04-09 07:55:09,080 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 6e82ea65bd89416ba123d8deb4c583f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897582 2023-04-09 07:55:09,080 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting d251bda3f1d84e7f94675a658f2caf7e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:09,080 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 489073fa9db54eae8fe5585df2e0d914, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:09,081 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting c4a5a528cb424261a35132ab5f18fcd0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:09,081 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting f1626be71f51415ba6c8e09d4ac42696, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:09,090 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 60a1e7a3b2b7470f369b184b87cd1dee#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:09,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139] to archive 2023-04-09 07:55:09,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:09,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/048a51cc9c4e4092893240c4631a54e3 2023-04-09 07:55:09,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/5d4812eb929743ffb69372e49be65034 2023-04-09 07:55:09,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/e0a6f1d523c140308259de663aae00e6 2023-04-09 07:55:09,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/b33d9c23d2a24add9cd9e8dd3382c4a7 2023-04-09 07:55:09,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/66d6d28e0e524f488bc4bc5a9d649139 2023-04-09 07:55:09,498 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del 2023-04-09 07:55:09,498 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del 2023-04-09 07:55:10,311 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99fac2678e8440f283b2da2771f7e3bb 2023-04-09 07:55:10,312 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/99fac2678e8440f283b2da2771f7e3bb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/99fac2678e8440f283b2da2771f7e3bb 2023-04-09 07:55:10,316 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99fac2678e8440f283b2da2771f7e3bb 2023-04-09 07:55:10,316 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 60a1e7a3b2b7470f369b184b87cd1dee/family1 of 60a1e7a3b2b7470f369b184b87cd1dee into 99fac2678e8440f283b2da2771f7e3bb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:10,316 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:10,317 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:10,317 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 60a1e7a3b2b7470f369b184b87cd1dee/family2 is initiating major compaction (all files) 2023-04-09 07:55:10,317 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:10,317 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:10,317 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 60a1e7a3b2b7470f369b184b87cd1dee/family2 in testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:10,317 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp, totalSize=25.4 K 2023-04-09 07:55:10,318 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 32d1220d842340d789e60950b9a1ba3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897582 2023-04-09 07:55:10,318 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting c797a98e70fa4ba1988dbd4005f84ea9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:10,318 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 338c2bebb8d54f2880050184268fa31e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:10,319 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 9050132042ca4d8296073af7e0057ac2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:10,319 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 8dcf4c9304a34944b4f5b0b92163f115, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:10,328 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 60a1e7a3b2b7470f369b184b87cd1dee#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:10,345 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696] to archive 2023-04-09 07:55:10,346 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:10,347 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/6e82ea65bd89416ba123d8deb4c583f2 2023-04-09 07:55:10,348 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/d251bda3f1d84e7f94675a658f2caf7e 2023-04-09 07:55:10,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/489073fa9db54eae8fe5585df2e0d914 2023-04-09 07:55:10,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/c4a5a528cb424261a35132ab5f18fcd0 2023-04-09 07:55:10,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/f1626be71f51415ba6c8e09d4ac42696 2023-04-09 07:55:10,665 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:10,737 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del 2023-04-09 07:55:10,737 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del 2023-04-09 07:55:11,550 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa1affda6d694f1eb8fa58ab2652a630 2023-04-09 07:55:11,550 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/aa1affda6d694f1eb8fa58ab2652a630 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/aa1affda6d694f1eb8fa58ab2652a630 2023-04-09 07:55:11,554 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aa1affda6d694f1eb8fa58ab2652a630 2023-04-09 07:55:11,555 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 60a1e7a3b2b7470f369b184b87cd1dee/family2 of 60a1e7a3b2b7470f369b184b87cd1dee into aa1affda6d694f1eb8fa58ab2652a630(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:11,555 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:11,555 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:11,555 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): d4ff53eca0e32f1940dd88a1cd07958a/family1 is initiating major compaction (all files) 2023-04-09 07:55:11,555 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:11,555 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:11,556 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of d4ff53eca0e32f1940dd88a1cd07958a/family1 in testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:11,556 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp, totalSize=26.7 K 2023-04-09 07:55:11,556 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting ab76dd4699904d23a90e5fb92ed66388, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897577 2023-04-09 07:55:11,557 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting c2fd7100e7464d1bbfa901c01d8cf756, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:11,557 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting d33553a5e93f408eacec7b222611058a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900507 2023-04-09 07:55:11,557 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 30ab0249d92e4fdcb426d4272ddba3be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:11,557 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 7a16bd7aedc44b3f82e54ad7a5b7d60e, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:11,567 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): d4ff53eca0e32f1940dd88a1cd07958a#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:11,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115] to archive 2023-04-09 07:55:11,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:11,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/32d1220d842340d789e60950b9a1ba3e 2023-04-09 07:55:11,648 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/c797a98e70fa4ba1988dbd4005f84ea9 2023-04-09 07:55:11,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/338c2bebb8d54f2880050184268fa31e 2023-04-09 07:55:11,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/9050132042ca4d8296073af7e0057ac2 2023-04-09 07:55:11,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/8dcf4c9304a34944b4f5b0b92163f115 2023-04-09 07:55:11,975 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del 2023-04-09 07:55:11,975 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del 2023-04-09 07:55:12,788 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1a7ee506bb1442eb382262b2f2f9983 2023-04-09 07:55:12,789 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/c1a7ee506bb1442eb382262b2f2f9983 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c1a7ee506bb1442eb382262b2f2f9983 2023-04-09 07:55:12,793 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1a7ee506bb1442eb382262b2f2f9983 2023-04-09 07:55:12,793 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d4ff53eca0e32f1940dd88a1cd07958a/family1 of d4ff53eca0e32f1940dd88a1cd07958a into c1a7ee506bb1442eb382262b2f2f9983(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:12,794 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:12,794 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:12,794 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): d4ff53eca0e32f1940dd88a1cd07958a/family2 is initiating major compaction (all files) 2023-04-09 07:55:12,794 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:12,794 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:12,794 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of d4ff53eca0e32f1940dd88a1cd07958a/family2 in testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:12,794 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp, totalSize=25.4 K 2023-04-09 07:55:12,795 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 670de79c0d7147e29301d37b7d05be6a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897577 2023-04-09 07:55:12,795 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting c60cda638dab443ba1daf428f043a481, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:12,795 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 4f2bfb0317dc40c397208cd03f0e4579, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900507 2023-04-09 07:55:12,795 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 24465f8bedd54e74b826892b610dcd3c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:12,796 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 6b9e5087285047d0983509a11dad8655, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-04-09 07:55:12,807 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): d4ff53eca0e32f1940dd88a1cd07958a#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:12,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e] to archive 2023-04-09 07:55:12,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:12,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/ab76dd4699904d23a90e5fb92ed66388 2023-04-09 07:55:12,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c2fd7100e7464d1bbfa901c01d8cf756 2023-04-09 07:55:12,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/d33553a5e93f408eacec7b222611058a 2023-04-09 07:55:12,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/30ab0249d92e4fdcb426d4272ddba3be 2023-04-09 07:55:12,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/7a16bd7aedc44b3f82e54ad7a5b7d60e 2023-04-09 07:55:13,215 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del 2023-04-09 07:55:13,216 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del 2023-04-09 07:55:14,028 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f3b69438936467287a953413997b424 2023-04-09 07:55:14,029 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/4f3b69438936467287a953413997b424 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f3b69438936467287a953413997b424 2023-04-09 07:55:14,031 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4f3b69438936467287a953413997b424 2023-04-09 07:55:14,032 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d4ff53eca0e32f1940dd88a1cd07958a/family2 of d4ff53eca0e32f1940dd88a1cd07958a into 4f3b69438936467287a953413997b424(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:14,032 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:14,037 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:14,040 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:14,044 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:14,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655] to archive 2023-04-09 07:55:14,045 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:14,045 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:14,047 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/670de79c0d7147e29301d37b7d05be6a 2023-04-09 07:55:14,048 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/c60cda638dab443ba1daf428f043a481 2023-04-09 07:55:14,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f2bfb0317dc40c397208cd03f0e4579 2023-04-09 07:55:14,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/24465f8bedd54e74b826892b610dcd3c 2023-04-09 07:55:14,051 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:55:14,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/6b9e5087285047d0983509a11dad8655 2023-04-09 07:55:14,084 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-04-09 07:55:14,087 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del 2023-04-09 07:55:14,090 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del 2023-04-09 07:55:14,093 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del 2023-04-09 07:55:14,093 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,093 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(139): Connect 0x18639285 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:14,098 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79e2a56f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:14,099 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:14,099 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:14,099 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:14,556 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3 2023-04-09 07:55:14,557 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,704 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-04-09 07:55:14,713 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:14,716 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60224, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:14,724 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:55:14,726 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37580, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:55:14,735 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/335d3f8eaa954c158494ea8cf457fe11 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-04-09 07:55:14,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/335d3f8eaa954c158494ea8cf457fe11}] 2023-04-09 07:55:14,740 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/335d3f8eaa954c158494ea8cf457fe11 for inclusion in d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:14,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-04-09 07:55:14,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last=1 2023-04-09 07:55:14,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:14,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/335d3f8eaa954c158494ea8cf457fe11 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__e9f0d4aeomc4icktb4ehimfufrd6nanf21dh8sqg9btnsr2p34ee26b18su9o7jp/family1/335d3f8eaa954c158494ea8cf457fe11 2023-04-09 07:55:14,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__e9f0d4aeomc4icktb4ehimfufrd6nanf21dh8sqg9btnsr2p34ee26b18su9o7jp/family1/335d3f8eaa954c158494ea8cf457fe11 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ 2023-04-09 07:55:14,745 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__e9f0d4aeomc4icktb4ehimfufrd6nanf21dh8sqg9btnsr2p34ee26b18su9o7jp/family1/335d3f8eaa954c158494ea8cf457fe11 into d4ff53eca0e32f1940dd88a1cd07958a/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ - updating store file list. 2023-04-09 07:55:14,747 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ into d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:14,748 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__e9f0d4aeomc4icktb4ehimfufrd6nanf21dh8sqg9btnsr2p34ee26b18su9o7jp/family1/335d3f8eaa954c158494ea8cf457fe11 into d4ff53eca0e32f1940dd88a1cd07958a/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_) 2023-04-09 07:55:14,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__e9f0d4aeomc4icktb4ehimfufrd6nanf21dh8sqg9btnsr2p34ee26b18su9o7jp/family1/335d3f8eaa954c158494ea8cf457fe11 2023-04-09 07:55:14,750 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,750 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:14,751 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 2023-04-09 07:55:14,752 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 2023-04-09 07:55:14,753 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f 2023-04-09 07:55:14,754 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 2023-04-09 07:55:14,754 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:14,754 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409' 2023-04-09 07:55:14,754 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:14,953 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4 2023-04-09 07:55:14,954 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,958 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4 2023-04-09 07:55:14,959 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,980 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/9594793e734a4ad6a42c877a229dd426 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-04-09 07:55:14,981 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/9594793e734a4ad6a42c877a229dd426}] 2023-04-09 07:55:14,983 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/9594793e734a4ad6a42c877a229dd426 for inclusion in 60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:14,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-04-09 07:55:14,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-09 07:55:14,985 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-04-09 07:55:14,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:14,986 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/587c7862ad1d4391ab2b2dc25e3f1bc1}] 2023-04-09 07:55:14,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/9594793e734a4ad6a42c877a229dd426 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__mt0c6h9nda7od775rlk8d07c0j0vbpvth06kq0j2hpk220s4s9gkgvvr2044nrv3/family1/9594793e734a4ad6a42c877a229dd426 2023-04-09 07:55:14,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 for inclusion in 9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:14,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__mt0c6h9nda7od775rlk8d07c0j0vbpvth06kq0j2hpk220s4s9gkgvvr2044nrv3/family1/9594793e734a4ad6a42c877a229dd426 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ 2023-04-09 07:55:14,989 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__mt0c6h9nda7od775rlk8d07c0j0vbpvth06kq0j2hpk220s4s9gkgvvr2044nrv3/family1/9594793e734a4ad6a42c877a229dd426 into 60a1e7a3b2b7470f369b184b87cd1dee/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ - updating store file list. 2023-04-09 07:55:14,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-04-09 07:55:14,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-09 07:55:14,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:14,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__rr07n51uiajoit10tgfqu7qd1mira4cab5vojehm1h57dn6jr6c89rmfdtop9m0j/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 2023-04-09 07:55:14,993 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ into 60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:14,993 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__mt0c6h9nda7od775rlk8d07c0j0vbpvth06kq0j2hpk220s4s9gkgvvr2044nrv3/family1/9594793e734a4ad6a42c877a229dd426 into 60a1e7a3b2b7470f369b184b87cd1dee/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_) 2023-04-09 07:55:14,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__rr07n51uiajoit10tgfqu7qd1mira4cab5vojehm1h57dn6jr6c89rmfdtop9m0j/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ 2023-04-09 07:55:14,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__mt0c6h9nda7od775rlk8d07c0j0vbpvth06kq0j2hpk220s4s9gkgvvr2044nrv3/family1/9594793e734a4ad6a42c877a229dd426 2023-04-09 07:55:14,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__rr07n51uiajoit10tgfqu7qd1mira4cab5vojehm1h57dn6jr6c89rmfdtop9m0j/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 into 9be6d4060ae57ce72cac4673686ad32d/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ - updating store file list. 2023-04-09 07:55:14,995 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,995 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:14,996 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e 2023-04-09 07:55:14,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ into 9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:14,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__rr07n51uiajoit10tgfqu7qd1mira4cab5vojehm1h57dn6jr6c89rmfdtop9m0j/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 into 9be6d4060ae57ce72cac4673686ad32d/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_) 2023-04-09 07:55:14,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__rr07n51uiajoit10tgfqu7qd1mira4cab5vojehm1h57dn6jr6c89rmfdtop9m0j/family1/587c7862ad1d4391ab2b2dc25e3f1bc1 2023-04-09 07:55:14,997 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 2023-04-09 07:55:14,998 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:14,998 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 2023-04-09 07:55:14,999 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:14,999 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 2023-04-09 07:55:14,999 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:14,999 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409' 2023-04-09 07:55:15,000 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:15,000 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d 2023-04-09 07:55:15,000 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 2023-04-09 07:55:15,001 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f 2023-04-09 07:55:15,002 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 2023-04-09 07:55:15,002 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:15,002 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409' 2023-04-09 07:55:15,002 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:15,002 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:15,003 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:15,003 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del] 2023-04-09 07:55:15,003 DEBUG [Listener at localhost/35977] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:15,004 DEBUG [Listener at localhost/35977] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409005a8225c30a4f8296cce67d119eec72_del 2023-04-09 07:55:15,004 INFO [Listener at localhost/35977] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-09 07:55:15,004 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del] 2023-04-09 07:55:15,005 DEBUG [Listener at localhost/35977] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:15,005 DEBUG [Listener at localhost/35977] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409e7ac5d1d717a48e881b19f3840c0e9f0_del 2023-04-09 07:55:15,006 INFO [Listener at localhost/35977] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-09 07:55:15,006 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del] 2023-04-09 07:55:15,006 DEBUG [Listener at localhost/35977] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:15,007 DEBUG [Listener at localhost/35977] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409dd178e0358a341448e35cc1a1bf05ecb_del 2023-04-09 07:55:15,007 INFO [Listener at localhost/35977] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-04-09 07:55:15,010 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:15,011 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:15,015 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:15,015 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:15,021 INFO [Listener at localhost/35977] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-04-09 07:55:15,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-04-09 07:55:15,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-04-09 07:55:15,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-09 07:55:15,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026915025"}]},"ts":"1681026915025"} 2023-04-09 07:55:15,026 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-04-09 07:55:15,028 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-04-09 07:55:15,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, UNASSIGN}] 2023-04-09 07:55:15,030 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, UNASSIGN 2023-04-09 07:55:15,030 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, UNASSIGN 2023-04-09 07:55:15,030 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, UNASSIGN 2023-04-09 07:55:15,030 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:15,030 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026915030"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026915030"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026915030"}]},"ts":"1681026915030"} 2023-04-09 07:55:15,030 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:15,030 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026915030"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026915030"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026915030"}]},"ts":"1681026915030"} 2023-04-09 07:55:15,031 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:15,031 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026915031"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026915031"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026915031"}]},"ts":"1681026915031"} 2023-04-09 07:55:15,032 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:15,032 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:15,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:15,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-09 07:55:15,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:15,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 60a1e7a3b2b7470f369b184b87cd1dee, disabling compactions & flushes 2023-04-09 07:55:15,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:15,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:15,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. after waiting 0 ms 2023-04-09 07:55:15,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:15,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-09 07:55:15,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:15,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:15,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:15,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:15,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d4ff53eca0e32f1940dd88a1cd07958a, disabling compactions & flushes 2023-04-09 07:55:15,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:15,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:15,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. after waiting 0 ms 2023-04-09 07:55:15,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:15,190 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=CLOSED 2023-04-09 07:55:15,190 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026915190"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026915190"}]},"ts":"1681026915190"} 2023-04-09 07:55:15,193 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-04-09 07:55:15,193 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245 in 158 msec 2023-04-09 07:55:15,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-09 07:55:15,194 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, UNASSIGN in 165 msec 2023-04-09 07:55:15,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:15,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:15,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:15,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:15,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9be6d4060ae57ce72cac4673686ad32d, disabling compactions & flushes 2023-04-09 07:55:15,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:15,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:15,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. after waiting 0 ms 2023-04-09 07:55:15,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:15,196 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=CLOSED 2023-04-09 07:55:15,196 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026915196"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026915196"}]},"ts":"1681026915196"} 2023-04-09 07:55:15,198 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-04-09 07:55:15,198 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245 in 165 msec 2023-04-09 07:55:15,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-04-09 07:55:15,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, UNASSIGN in 170 msec 2023-04-09 07:55:15,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:15,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:15,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:15,201 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=CLOSED 2023-04-09 07:55:15,201 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026915201"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026915201"}]},"ts":"1681026915201"} 2023-04-09 07:55:15,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-04-09 07:55:15,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245 in 170 msec 2023-04-09 07:55:15,204 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-04-09 07:55:15,204 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, UNASSIGN in 175 msec 2023-04-09 07:55:15,205 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026915205"}]},"ts":"1681026915205"} 2023-04-09 07:55:15,205 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-04-09 07:55:15,207 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-04-09 07:55:15,208 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 185 msec 2023-04-09 07:55:15,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-09 07:55:15,327 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-04-09 07:55:15,336 INFO [Listener at localhost/35977] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:15,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:15,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1681026915337 (current time:1681026915337). 2023-04-09 07:55:15,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-04-09 07:55:15,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] zookeeper.ReadOnlyZKClient(139): Connect 0x140f918f to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:15,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18567449, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:15,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:15,344 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60238, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:15,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x140f918f to 127.0.0.1:57867 2023-04-09 07:55:15,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:15,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-04-09 07:55:15,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:15,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:15,348 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:15,349 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1681027515349 2023-04-09 07:55:15,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:15,349 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1681026903895-1681026915336 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-04-09 07:55:15,349 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:15,349 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-04-09 07:55:15,449 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:15,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:15,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:15,450 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-04-09 07:55:15,650 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:15,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:15,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:15,651 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-04-09 07:55:15,755 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-04-09 07:55:15,757 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-09 07:55:15,756 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/350db1ee5e6f427b907a8ac75d7bcd87, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_] hfiles 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/350db1ee5e6f427b907a8ac75d7bcd87 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/99fac2678e8440f283b2da2771f7e3bb] hfiles 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c1a7ee506bb1442eb382262b2f2f9983] hfiles 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ 2023-04-09 07:55:15,759 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3] hfiles 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c1a7ee506bb1442eb382262b2f2f9983 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/99fac2678e8440f283b2da2771f7e3bb 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/80998ca50f1246d5afea4df7caeb3443] hfiles 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/80998ca50f1246d5afea4df7caeb3443 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f3b69438936467287a953413997b424] hfiles 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/aa1affda6d694f1eb8fa58ab2652a630] hfiles 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f3b69438936467287a953413997b424 2023-04-09 07:55:15,760 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/aa1affda6d694f1eb8fa58ab2652a630 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del] hfiles 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304092b327943ea96442296e459d343084284 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304097eddb4534f294c54b4a741367fa61b8f 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409b283d6527f794312baa619917bafe290 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230409e3eff47bd0a340078abfe9c23b4ebb3e 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202304096f447abd17da48528f957507af0e4cec 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del 2023-04-09 07:55:15,772 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409bc647e7c35af49c391d71191195792ce 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409da16317b3a36448980e706d677c983d4 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230409fe497862c8b047fdb4c0cdfc86522c94 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099a959d34f7b448499997448f576f3f25 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202304099b7e3f93c2ab4a699ad61bb3f97b47ca 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409a5920a19fc43410c9221825c9eca39d4 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409cc074e9ca89f40dfa92bbf19b93cf440 2023-04-09 07:55:15,773 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del 2023-04-09 07:55:15,952 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:15,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:15,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:15,953 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-04-09 07:55:16,188 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-04-09 07:55:16,189 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:16,189 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:16,453 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:16,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:16,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-04-09 07:55:16,454 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-04-09 07:55:16,612 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1681026903895-1681026915336 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:17,426 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1681026903895-1681026915336 of table testCompactionWithHFileLink completed 2023-04-09 07:55:17,426 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:17,426 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:17,426 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1681026903895-1681026915336 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1681026915349Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1681026915755 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1681026903895-1681026915336 at 1681026916189 (+434 ms)Loading Region manifests for hbase-failsafe-snaptb-1681026903895-1681026915336 at 1681026916189Writing data manifest for hbase-failsafe-snaptb-1681026903895-1681026915336 at 1681026916192 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1681026903895-1681026915336 at 1681026916599 (+407 ms)Snapshot hbase-failsafe-snaptb-1681026903895-1681026915336 of table testCompactionWithHFileLink completed at 1681026917426 (+827 ms) 2023-04-09 07:55:17,428 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-04-09 07:55:17,429 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0810 sec 2023-04-09 07:55:17,454 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-04-09 07:55:17,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-04-09 07:55:17,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1681026903895-1681026915336 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-04-09 07:55:17,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1681026903895" table: "testCompactionWithHFileLink" creation_time: 1681026903907 type: FLUSH version: 2 ttl: 0 ) 2023-04-09 07:55:17,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1681026903895 as table=testCompactionWithHFileLink 2023-04-09 07:55:17,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-09 07:55:17,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-09 07:55:17,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-09 07:55:17,899 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-04-09 07:55:17,900 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-04-09 07:55:17,900 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-04-09 07:55:17,901 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-04-09 07:55:17,906 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1681026903895" table: "testCompactionWithHFileLink" creation_time: 1681026903907 type: FLUSH version: 2 ttl: 0 2023-04-09 07:55:17,907 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink 2023-04-09 07:55:17,912 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-04-09 07:55:17,915 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202304090f5b578570ba4be88d5cad97843acdd3 2023-04-09 07:55:17,916 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409b4580c0f0bd14ef49803c7f187e4dcb4 2023-04-09 07:55:17,918 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202304094b7ef9d7a8e94f11b634d3efcc5ee9c4 2023-04-09 07:55:17,918 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,921 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,924 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,927 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,929 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,932 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,935 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,938 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,940 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,943 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,946 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,948 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 of snapshot=snaptb-1681026903895 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-04-09 07:55:17,952 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230409d5e19d798a7c453da9a1b61cf13d2cd4_del 2023-04-09 07:55:17,953 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023040970e5e221f63744c5a8bd96a1d82da80d_del 2023-04-09 07:55:17,954 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202304093ed7fd7e78b84d6ab11a9a8891f16204_del 2023-04-09 07:55:17,955 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:17,955 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:17,955 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:17,958 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/16c02571b40d42cfa16210496837ab1e_SeqId_29_ 2023-04-09 07:55:17,958 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/4611ef18585f4b2580a0e3a933570142_SeqId_29_ 2023-04-09 07:55:17,958 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/4c1aca349ef94eff9a77d70690cea2f5_SeqId_29_ 2023-04-09 07:55:17,959 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/99fac2678e8440f283b2da2771f7e3bb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/99fac2678e8440f283b2da2771f7e3bb 2023-04-09 07:55:17,959 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6e82ea65bd89416ba123d8deb4c583f2 of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,959 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/350db1ee5e6f427b907a8ac75d7bcd87 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/350db1ee5e6f427b907a8ac75d7bcd87 2023-04-09 07:55:17,959 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc3980414a4348f6a3429b4c34391e56 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,960 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c1a7ee506bb1442eb382262b2f2f9983 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/c1a7ee506bb1442eb382262b2f2f9983 2023-04-09 07:55:17,960 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ab76dd4699904d23a90e5fb92ed66388 of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,963 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d251bda3f1d84e7f94675a658f2caf7e of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,964 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5987d3843a9541ec9705d9216fa5d916 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,972 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c2fd7100e7464d1bbfa901c01d8cf756 of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,972 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 489073fa9db54eae8fe5585df2e0d914 of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,973 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0512b4adaf1b4014a3bf577864b0cd49 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,976 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d33553a5e93f408eacec7b222611058a of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,976 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4a5a528cb424261a35132ab5f18fcd0 of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,977 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8530d077b2fb44e8b1bf6dbdc609356d of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,980 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 30ab0249d92e4fdcb426d4272ddba3be of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,981 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/aa1affda6d694f1eb8fa58ab2652a630 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/aa1affda6d694f1eb8fa58ab2652a630 2023-04-09 07:55:17,981 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 32d1220d842340d789e60950b9a1ba3e of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,983 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/80998ca50f1246d5afea4df7caeb3443 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/80998ca50f1246d5afea4df7caeb3443 2023-04-09 07:55:17,983 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 048a51cc9c4e4092893240c4631a54e3 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,988 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c797a98e70fa4ba1988dbd4005f84ea9 of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,989 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5d4812eb929743ffb69372e49be65034 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,989 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f3b69438936467287a953413997b424 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4f3b69438936467287a953413997b424 2023-04-09 07:55:17,989 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 670de79c0d7147e29301d37b7d05be6a of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,992 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 338c2bebb8d54f2880050184268fa31e of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:17,993 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e0a6f1d523c140308259de663aae00e6 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,994 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c60cda638dab443ba1daf428f043a481 of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:17,999 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b33d9c23d2a24add9cd9e8dd3382c4a7 of snapshot=snaptb-1681026903895 to region=9be6d4060ae57ce72cac4673686ad32d table=testCompactionWithHFileLink 2023-04-09 07:55:17,999 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9050132042ca4d8296073af7e0057ac2 of snapshot=snaptb-1681026903895 to region=60a1e7a3b2b7470f369b184b87cd1dee table=testCompactionWithHFileLink 2023-04-09 07:55:18,003 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4f2bfb0317dc40c397208cd03f0e4579 of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:18,007 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 24465f8bedd54e74b826892b610dcd3c of snapshot=snaptb-1681026903895 to region=d4ff53eca0e32f1940dd88a1cd07958a table=testCompactionWithHFileLink 2023-04-09 07:55:18,010 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1681026903895" table: "testCompactionWithHFileLink" creation_time: 1681026903907 type: FLUSH version: 2 ttl: 0 2023-04-09 07:55:18,011 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1681026918011"}]},"ts":"9223372036854775807"} 2023-04-09 07:55:18,011 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1681026918011"}]},"ts":"9223372036854775807"} 2023-04-09 07:55:18,011 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1681026918011"}]},"ts":"9223372036854775807"} 2023-04-09 07:55:18,014 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-09 07:55:18,014 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 60a1e7a3b2b7470f369b184b87cd1dee, NAME => 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 9be6d4060ae57ce72cac4673686ad32d, NAME => 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => d4ff53eca0e32f1940dd88a1cd07958a, NAME => 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.', STARTKEY => '', ENDKEY => '1'}] 2023-04-09 07:55:18,014 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918012"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026918012"}]},"ts":"1681026918012"} 2023-04-09 07:55:18,014 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026918012"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026918012"}]},"ts":"1681026918012"} 2023-04-09 07:55:18,014 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918012"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026918012"}]},"ts":"1681026918012"} 2023-04-09 07:55:18,016 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-09 07:55:18,016 INFO [PEWorker-4] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-04-09 07:55:18,016 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 60a1e7a3b2b7470f369b184b87cd1dee, NAME => 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 9be6d4060ae57ce72cac4673686ad32d, NAME => 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => d4ff53eca0e32f1940dd88a1cd07958a, NAME => 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.', STARTKEY => '', ENDKEY => '1'}] 2023-04-09 07:55:18,017 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1681026903895 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-04-09 07:55:18,018 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1681026903895" table: "testCompactionWithHFileLink" creation_time: 1681026903907 type: FLUSH version: 2 ttl: 0 ) in 536 msec 2023-04-09 07:55:18,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-09 07:55:18,097 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-04-09 07:55:18,097 INFO [Listener at localhost/35977] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:18,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1681026903895-1681026915336" 2023-04-09 07:55:18,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1681026903895-1681026915336 2023-04-09 07:55:18,105 INFO [Listener at localhost/35977] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-04-09 07:55:18,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-04-09 07:55:18,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-04-09 07:55:18,117 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-04-09 07:55:18,117 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026918117"}]},"ts":"1681026918117"} 2023-04-09 07:55:18,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-09 07:55:18,118 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-04-09 07:55:18,121 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-04-09 07:55:18,121 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN}] 2023-04-09 07:55:18,122 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN 2023-04-09 07:55:18,122 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN 2023-04-09 07:55:18,123 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN 2023-04-09 07:55:18,123 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-09 07:55:18,123 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-09 07:55:18,123 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-04-09 07:55:18,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-09 07:55:18,275 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,275 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,275 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026918275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026918275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026918275"}]},"ts":"1681026918275"} 2023-04-09 07:55:18,275 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,275 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026918275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026918275"}]},"ts":"1681026918275"} 2023-04-09 07:55:18,275 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918275"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026918275"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026918275"}]},"ts":"1681026918275"} 2023-04-09 07:55:18,276 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:18,277 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:18,277 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=100, state=RUNNABLE; OpenRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:18,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-09 07:55:18,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d4ff53eca0e32f1940dd88a1cd07958a, NAME => 'testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.', STARTKEY => '', ENDKEY => '1'} 2023-04-09 07:55:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,432 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,433 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:18,433 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:18,433 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4ff53eca0e32f1940dd88a1cd07958a columnFamilyName family1 2023-04-09 07:55:18,444 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be 2023-04-09 07:55:18,448 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388 2023-04-09 07:55:18,453 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756 2023-04-09 07:55:18,457 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a 2023-04-09 07:55:18,457 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(310): Store=d4ff53eca0e32f1940dd88a1cd07958a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,457 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,458 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2 2023-04-09 07:55:18,458 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2 2023-04-09 07:55:18,458 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d4ff53eca0e32f1940dd88a1cd07958a columnFamilyName family2 2023-04-09 07:55:18,464 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c 2023-04-09 07:55:18,468 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579 2023-04-09 07:55:18,472 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a 2023-04-09 07:55:18,477 DEBUG [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481 2023-04-09 07:55:18,477 INFO [StoreOpener-d4ff53eca0e32f1940dd88a1cd07958a-1] regionserver.HStore(310): Store=d4ff53eca0e32f1940dd88a1cd07958a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:18,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:55:18,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d4ff53eca0e32f1940dd88a1cd07958a; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64887951, jitterRate=-0.03309418261051178}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:18,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:18,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., pid=102, masterSystemTime=1681026918428 2023-04-09 07:55:18,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:18,483 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-09 07:55:18,483 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-09 07:55:18,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-09 07:55:18,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:18,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:18,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:18,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 60a1e7a3b2b7470f369b184b87cd1dee, NAME => 'testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.', STARTKEY => '2', ENDKEY => ''} 2023-04-09 07:55:18,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,485 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=d4ff53eca0e32f1940dd88a1cd07958a, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:18,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,485 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918485"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026918485"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026918485"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026918485"}]},"ts":"1681026918485"} 2023-04-09 07:55:18,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,485 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-09 07:55:18,486 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1912): d4ff53eca0e32f1940dd88a1cd07958a/family1 is initiating minor compaction (all files) 2023-04-09 07:55:18,486 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d4ff53eca0e32f1940dd88a1cd07958a/family1 in testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:18,486 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp, totalSize=21.6 K 2023-04-09 07:55:18,486 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-09 07:55:18,486 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1912): d4ff53eca0e32f1940dd88a1cd07958a/family2 is initiating minor compaction (all files) 2023-04-09 07:55:18,486 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d4ff53eca0e32f1940dd88a1cd07958a/family2 in testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:55:18,486 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,487 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp, totalSize=20.4 K 2023-04-09 07:55:18,487 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897577 2023-04-09 07:55:18,487 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:18,488 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:18,488 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897577 2023-04-09 07:55:18,488 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:18,488 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a1e7a3b2b7470f369b184b87cd1dee columnFamilyName family1 2023-04-09 07:55:18,488 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900507 2023-04-09 07:55:18,488 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:18,490 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-04-09 07:55:18,490 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure d4ff53eca0e32f1940dd88a1cd07958a, server=jenkins-hbase4.apache.org,34907,1681026816245 in 212 msec 2023-04-09 07:55:18,491 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=d4ff53eca0e32f1940dd88a1cd07958a, ASSIGN in 369 msec 2023-04-09 07:55:18,492 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:18,492 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900507 2023-04-09 07:55:18,494 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:18,509 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914 2023-04-09 07:55:18,517 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2 2023-04-09 07:55:18,525 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0 2023-04-09 07:55:18,539 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e 2023-04-09 07:55:18,540 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(310): Store=60a1e7a3b2b7470f369b184b87cd1dee/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,540 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] throttle.PressureAwareThroughputController(145): d4ff53eca0e32f1940dd88a1cd07958a#family2#compaction#114 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:18,540 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,544 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): d4ff53eca0e32f1940dd88a1cd07958a#family1#compaction#113 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:18,544 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2 2023-04-09 07:55:18,544 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2 2023-04-09 07:55:18,544 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60a1e7a3b2b7470f369b184b87cd1dee columnFamilyName family2 2023-04-09 07:55:18,565 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e 2023-04-09 07:55:18,574 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e 2023-04-09 07:55:18,578 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2 2023-04-09 07:55:18,582 DEBUG [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9 2023-04-09 07:55:18,582 INFO [StoreOpener-60a1e7a3b2b7470f369b184b87cd1dee-1] regionserver.HStore(310): Store=60a1e7a3b2b7470f369b184b87cd1dee/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:18,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:55:18,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 60a1e7a3b2b7470f369b184b87cd1dee; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63418917, jitterRate=-0.05498449504375458}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:18,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:18,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., pid=103, masterSystemTime=1681026918428 2023-04-09 07:55:18,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-09 07:55:18,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-09 07:55:18,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:18,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:18,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:18,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9be6d4060ae57ce72cac4673686ad32d, NAME => 'testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.', STARTKEY => '1', ENDKEY => '2'} 2023-04-09 07:55:18,589 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=60a1e7a3b2b7470f369b184b87cd1dee, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:18,589 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1681026918589"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026918589"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026918589"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026918589"}]},"ts":"1681026918589"} 2023-04-09 07:55:18,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,590 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,591 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:18,591 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:18,591 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9be6d4060ae57ce72cac4673686ad32d columnFamilyName family1 2023-04-09 07:55:18,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=100 2023-04-09 07:55:18,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=100, state=SUCCESS; OpenRegionProcedure 60a1e7a3b2b7470f369b184b87cd1dee, server=jenkins-hbase4.apache.org,34907,1681026816245 in 313 msec 2023-04-09 07:55:18,592 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=60a1e7a3b2b7470f369b184b87cd1dee, ASSIGN in 470 msec 2023-04-09 07:55:18,599 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49 2023-04-09 07:55:18,604 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916 2023-04-09 07:55:18,609 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d 2023-04-09 07:55:18,614 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56 2023-04-09 07:55:18,614 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(310): Store=9be6d4060ae57ce72cac4673686ad32d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,615 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,615 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2 2023-04-09 07:55:18,615 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2 2023-04-09 07:55:18,616 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9be6d4060ae57ce72cac4673686ad32d columnFamilyName family2 2023-04-09 07:55:18,622 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3 2023-04-09 07:55:18,627 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034 2023-04-09 07:55:18,633 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7 2023-04-09 07:55:18,638 DEBUG [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(539): loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6 2023-04-09 07:55:18,638 INFO [StoreOpener-9be6d4060ae57ce72cac4673686ad32d-1] regionserver.HStore(310): Store=9be6d4060ae57ce72cac4673686ad32d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:18,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:18,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:55:18,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9be6d4060ae57ce72cac4673686ad32d; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74386537, jitterRate=0.10844577848911285}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:18,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:18,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., pid=101, masterSystemTime=1681026918428 2023-04-09 07:55:18,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-04-09 07:55:18,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-04-09 07:55:18,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:18,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:18,645 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=9be6d4060ae57ce72cac4673686ad32d, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:18,646 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026918645"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026918645"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026918645"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026918645"}]},"ts":"1681026918645"} 2023-04-09 07:55:18,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-04-09 07:55:18,648 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 9be6d4060ae57ce72cac4673686ad32d, server=jenkins-hbase4.apache.org,34907,1681026816245 in 371 msec 2023-04-09 07:55:18,649 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-04-09 07:55:18,649 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=9be6d4060ae57ce72cac4673686ad32d, ASSIGN in 527 msec 2023-04-09 07:55:18,649 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026918649"}]},"ts":"1681026918649"} 2023-04-09 07:55:18,650 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-04-09 07:55:18,652 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-04-09 07:55:18,653 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 542 msec 2023-04-09 07:55:18,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-04-09 07:55:18,720 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-04-09 07:55:18,733 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:18,745 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:18,754 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:18,755 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:18,762 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:55:18,777 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-04-09 07:55:18,777 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:18,777 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(139): Connect 0x32f4cb3b to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:18,782 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27841654, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:18,782 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:18,782 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:18,782 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:19,232 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409a461b9b05a864ebab7720bb808cc090a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230409a461b9b05a864ebab7720bb808cc090a 2023-04-09 07:55:19,232 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409eccb9981aff04f39b6c883ff49411681 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230409eccb9981aff04f39b6c883ff49411681 2023-04-09 07:55:19,232 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,232 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,245 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:19,249 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60252, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:19,254 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:55:19,257 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37590, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:55:19,269 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-04-09 07:55:19,270 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/685f8c8b0a2d454b8d4f89f9bd3bb096}] 2023-04-09 07:55:19,270 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a81a79d6f384481eb7771afaf6e402c0 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-04-09 07:55:19,271 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a81a79d6f384481eb7771afaf6e402c0}] 2023-04-09 07:55:19,276 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 for inclusion in 9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:19,276 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a81a79d6f384481eb7771afaf6e402c0 for inclusion in 60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:19,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-04-09 07:55:19,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-09 07:55:19,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:19,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-04-09 07:55:19,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-09 07:55:19,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:19,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__731pn9v6b0amt9sh3lil72vimvcpsfh060gv1toi6kbi9b9ic6ct5qtuct02tdrr/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 2023-04-09 07:55:19,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a81a79d6f384481eb7771afaf6e402c0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__el8s7fmsrlcituqun2qoa0nckauj5v072j8s4tl91mhc251lk85q0gal0705p3hg/family1/a81a79d6f384481eb7771afaf6e402c0 2023-04-09 07:55:19,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__731pn9v6b0amt9sh3lil72vimvcpsfh060gv1toi6kbi9b9ic6ct5qtuct02tdrr/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_ 2023-04-09 07:55:19,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__el8s7fmsrlcituqun2qoa0nckauj5v072j8s4tl91mhc251lk85q0gal0705p3hg/family1/a81a79d6f384481eb7771afaf6e402c0 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_ 2023-04-09 07:55:19,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__731pn9v6b0amt9sh3lil72vimvcpsfh060gv1toi6kbi9b9ic6ct5qtuct02tdrr/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 into 9be6d4060ae57ce72cac4673686ad32d/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_ - updating store file list. 2023-04-09 07:55:19,283 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__el8s7fmsrlcituqun2qoa0nckauj5v072j8s4tl91mhc251lk85q0gal0705p3hg/family1/a81a79d6f384481eb7771afaf6e402c0 into 60a1e7a3b2b7470f369b184b87cd1dee/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_ - updating store file list. 2023-04-09 07:55:19,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_ into 9be6d4060ae57ce72cac4673686ad32d/family1 2023-04-09 07:55:19,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__731pn9v6b0amt9sh3lil72vimvcpsfh060gv1toi6kbi9b9ic6ct5qtuct02tdrr/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 into 9be6d4060ae57ce72cac4673686ad32d/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_) 2023-04-09 07:55:19,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_ into 60a1e7a3b2b7470f369b184b87cd1dee/family1 2023-04-09 07:55:19,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__el8s7fmsrlcituqun2qoa0nckauj5v072j8s4tl91mhc251lk85q0gal0705p3hg/family1/a81a79d6f384481eb7771afaf6e402c0 into 60a1e7a3b2b7470f369b184b87cd1dee/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_) 2023-04-09 07:55:19,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__731pn9v6b0amt9sh3lil72vimvcpsfh060gv1toi6kbi9b9ic6ct5qtuct02tdrr/family1/685f8c8b0a2d454b8d4f89f9bd3bb096 2023-04-09 07:55:19,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:19,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__el8s7fmsrlcituqun2qoa0nckauj5v072j8s4tl91mhc251lk85q0gal0705p3hg/family1/a81a79d6f384481eb7771afaf6e402c0 2023-04-09 07:55:19,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-04-09 07:55:19,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.HStore(1912): 9be6d4060ae57ce72cac4673686ad32d/family1 is initiating minor compaction (all files) 2023-04-09 07:55:19,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@639f15fe; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-09 07:55:19,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:19,290 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,291 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:19,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-04-09 07:55:19,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): 60a1e7a3b2b7470f369b184b87cd1dee/family1 is initiating minor compaction (all files) 2023-04-09 07:55:19,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5c921b3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-04-09 07:55:19,292 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304090656ec4768f24eb3852367c3eb66818d 2023-04-09 07:55:19,293 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,293 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304093417051303554cb492c0a0256aafe213 2023-04-09 07:55:19,293 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:19,294 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202304094a16d466f1f14e9292235493494e1d4f 2023-04-09 07:55:19,295 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023040957fcdb3bc4ec4138848ac8fb8702f75e 2023-04-09 07:55:19,295 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023040969d6fcd6fd8548288264e785306d2b17 2023-04-09 07:55:19,295 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:19,295 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ed11ca2c1b47444185d84c1ee522a770 2023-04-09 07:55:19,295 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230409' 2023-04-09 07:55:19,296 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:19,296 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ed67ed903f664b55a38300a2c85b7bd0 2023-04-09 07:55:19,297 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230409ff423cb5f17e411d98ce2707f91973a2 2023-04-09 07:55:19,297 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:19,297 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230409' 2023-04-09 07:55:19,298 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:19,373 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family2/4733515c4011439f8cafcb1ddec7c5ab as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/4733515c4011439f8cafcb1ddec7c5ab 2023-04-09 07:55:19,377 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/.tmp/family1/213292cbff16430296b4a57f5594e1d7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/213292cbff16430296b4a57f5594e1d7 2023-04-09 07:55:19,377 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d4ff53eca0e32f1940dd88a1cd07958a/family2 of d4ff53eca0e32f1940dd88a1cd07958a into 4733515c4011439f8cafcb1ddec7c5ab(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:19,378 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:19,378 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., storeName=d4ff53eca0e32f1940dd88a1cd07958a/family2, priority=12, startTime=1681026918483; duration=0sec 2023-04-09 07:55:19,378 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-04-09 07:55:19,378 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 9be6d4060ae57ce72cac4673686ad32d/family1 in testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:19,378 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp, totalSize=28.9 K 2023-04-09 07:55:19,379 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897581 2023-04-09 07:55:19,379 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:19,380 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:19,381 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in d4ff53eca0e32f1940dd88a1cd07958a/family1 of d4ff53eca0e32f1940dd88a1cd07958a into 213292cbff16430296b4a57f5594e1d7(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:19,381 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:19,381 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:19,381 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., storeName=d4ff53eca0e32f1940dd88a1cd07958a/family1, priority=12, startTime=1681026918482; duration=0sec 2023-04-09 07:55:19,381 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-04-09 07:55:19,381 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting f665ed52f94549059ab4d25040d0c561_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1681026897581 2023-04-09 07:55:19,381 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 60a1e7a3b2b7470f369b184b87cd1dee/family1 in testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:19,381 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp, totalSize=28.9 K 2023-04-09 07:55:19,382 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897582 2023-04-09 07:55:19,382 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:19,383 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:19,384 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:19,384 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting fd6e9db437b140dcbe42beabb66ec312_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1681026897582 2023-04-09 07:55:19,395 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] throttle.PressureAwareThroughputController(145): 9be6d4060ae57ce72cac4673686ad32d#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:19,400 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 60a1e7a3b2b7470f369b184b87cd1dee#family1#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:19,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be] to archive 2023-04-09 07:55:19,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:19,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-ab76dd4699904d23a90e5fb92ed66388 2023-04-09 07:55:19,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c2fd7100e7464d1bbfa901c01d8cf756 2023-04-09 07:55:19,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-d33553a5e93f408eacec7b222611058a 2023-04-09 07:55:19,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-30ab0249d92e4fdcb426d4272ddba3be 2023-04-09 07:55:19,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c] to archive 2023-04-09 07:55:19,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:19,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-670de79c0d7147e29301d37b7d05be6a 2023-04-09 07:55:19,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-c60cda638dab443ba1daf428f043a481 2023-04-09 07:55:19,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-4f2bfb0317dc40c397208cd03f0e4579 2023-04-09 07:55:19,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family2/testCompactionWithHFileLink=d4ff53eca0e32f1940dd88a1cd07958a-24465f8bedd54e74b826892b610dcd3c 2023-04-09 07:55:19,631 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409888b1b9108eb4257bd24eb455bc1bc90 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230409888b1b9108eb4257bd24eb455bc1bc90 2023-04-09 07:55:19,632 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,655 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/285180f8a26d4ad0a8d825d77a4ec9fb first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-04-09 07:55:19,655 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=32 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/285180f8a26d4ad0a8d825d77a4ec9fb}] 2023-04-09 07:55:19,657 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/285180f8a26d4ad0a8d825d77a4ec9fb for inclusion in d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:19,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-04-09 07:55:19,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last=1 2023-04-09 07:55:19,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:55:19,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409/family1/285180f8a26d4ad0a8d825d77a4ec9fb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__v81p2ohhbqegrde76m44jii39k2paf4ekks8lkb93e6l3i5uaaf9rit0t20jpana/family1/285180f8a26d4ad0a8d825d77a4ec9fb 2023-04-09 07:55:19,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__v81p2ohhbqegrde76m44jii39k2paf4ekks8lkb93e6l3i5uaaf9rit0t20jpana/family1/285180f8a26d4ad0a8d825d77a4ec9fb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/cea082d273804015ac840e713ebe94d1_SeqId_36_ 2023-04-09 07:55:19,662 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__v81p2ohhbqegrde76m44jii39k2paf4ekks8lkb93e6l3i5uaaf9rit0t20jpana/family1/285180f8a26d4ad0a8d825d77a4ec9fb into d4ff53eca0e32f1940dd88a1cd07958a/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/cea082d273804015ac840e713ebe94d1_SeqId_36_ - updating store file list. 2023-04-09 07:55:19,664 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/cea082d273804015ac840e713ebe94d1_SeqId_36_ into d4ff53eca0e32f1940dd88a1cd07958a/family1 2023-04-09 07:55:19,664 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__v81p2ohhbqegrde76m44jii39k2paf4ekks8lkb93e6l3i5uaaf9rit0t20jpana/family1/285180f8a26d4ad0a8d825d77a4ec9fb into d4ff53eca0e32f1940dd88a1cd07958a/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/family1/cea082d273804015ac840e713ebe94d1_SeqId_36_) 2023-04-09 07:55:19,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testCompactionWithHFileLink__v81p2ohhbqegrde76m44jii39k2paf4ekks8lkb93e6l3i5uaaf9rit0t20jpana/family1/285180f8a26d4ad0a8d825d77a4ec9fb 2023-04-09 07:55:19,666 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,667 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:19,668 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202304090b851fda909d4b83a9bda691e3141dd1 2023-04-09 07:55:19,669 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023040965ba0aee55d14f45afa28b5df4366cc9 2023-04-09 07:55:19,670 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202304097d595cfa4f7b472b8ef82e706b55434f 2023-04-09 07:55:19,670 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230409fd21c8506bb446a6a7ac6a875d5e4504 2023-04-09 07:55:19,670 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-04-09 07:55:19,670 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230409' 2023-04-09 07:55:19,671 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230409 for table='testCompactionWithHFileLink' and column='family1' 2023-04-09 07:55:19,671 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-04-09 07:55:19,674 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:19,676 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:19,679 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:19,679 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:19,685 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:19,686 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:19,702 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=287 (was 260) - Thread LEAK? -, OpenFileDescriptor=759 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=286 (was 311), ProcessCount=169 (was 169), AvailableMemoryMB=1479 (was 1627) 2023-04-09 07:55:19,714 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=287, OpenFileDescriptor=759, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=169, AvailableMemoryMB=1478 2023-04-09 07:55:19,718 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-04-09 07:55:19,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-04-09 07:55:19,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-04-09 07:55:19,725 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-09 07:55:19,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 7 msec 2023-04-09 07:55:19,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-04-09 07:55:19,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-09 07:55:19,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-04-09 07:55:19,827 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:55:19,828 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-04-09 07:55:19,828 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:55:19,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed empty. 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00 empty. 2023-04-09 07:55:19,830 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678 empty. 2023-04-09 07:55:19,831 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:19,831 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:19,831 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:19,831 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-04-09 07:55:19,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:20,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:20,237 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-04-09 07:55:20,238 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 351ba7e24003baba73ac0a889f713678, NAME => 'ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:20,238 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 410e3eed8a4051a7a5cebb43c08ccfed, NAME => 'ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:20,238 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => c669f14846d655d587709ca8bd4b7e00, NAME => 'ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:20,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:20,611 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family1/5f8f3a778e5a47dd9014712d23c00c3a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/5f8f3a778e5a47dd9014712d23c00c3a 2023-04-09 07:55:20,614 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family1/a246943f18344d8c86cc7888f6e9b7f4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/a246943f18344d8c86cc7888f6e9b7f4 2023-04-09 07:55:20,615 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 9be6d4060ae57ce72cac4673686ad32d/family1 of 9be6d4060ae57ce72cac4673686ad32d into 5f8f3a778e5a47dd9014712d23c00c3a(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:20,615 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:20,615 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1681026919288; duration=1sec 2023-04-09 07:55:20,615 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-04-09 07:55:20,615 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-09 07:55:20,617 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-09 07:55:20,617 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1912): 60a1e7a3b2b7470f369b184b87cd1dee/family2 is initiating minor compaction (all files) 2023-04-09 07:55:20,617 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 60a1e7a3b2b7470f369b184b87cd1dee/family2 in testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:55:20,617 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 60a1e7a3b2b7470f369b184b87cd1dee/family1 of 60a1e7a3b2b7470f369b184b87cd1dee into a246943f18344d8c86cc7888f6e9b7f4(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:20,618 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp, totalSize=20.4 K 2023-04-09 07:55:20,618 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:20,618 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1681026919291; duration=1sec 2023-04-09 07:55:20,618 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-04-09 07:55:20,618 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-09 07:55:20,618 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897582 2023-04-09 07:55:20,619 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:20,619 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:20,619 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-04-09 07:55:20,619 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1912): 9be6d4060ae57ce72cac4673686ad32d/family2 is initiating minor compaction (all files) 2023-04-09 07:55:20,619 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 9be6d4060ae57ce72cac4673686ad32d/family2 in testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:55:20,620 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp, totalSize=20.4 K 2023-04-09 07:55:20,620 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:20,620 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026897581 2023-04-09 07:55:20,621 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026899008 2023-04-09 07:55:20,621 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026900508 2023-04-09 07:55:20,621 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026902384 2023-04-09 07:55:20,635 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] throttle.PressureAwareThroughputController(145): 60a1e7a3b2b7470f369b184b87cd1dee#family2#compaction#117 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:20,636 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9be6d4060ae57ce72cac4673686ad32d#family2#compaction#118 average throughput is 0.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 351ba7e24003baba73ac0a889f713678, disabling compactions & flushes 2023-04-09 07:55:20,644 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 410e3eed8a4051a7a5cebb43c08ccfed, disabling compactions & flushes 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,644 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing c669f14846d655d587709ca8bd4b7e00, disabling compactions & flushes 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. after waiting 0 ms 2023-04-09 07:55:20,645 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,644 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. after waiting 0 ms 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. after waiting 0 ms 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,645 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,645 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:20,645 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,645 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:20,647 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:55:20,647 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1681026920647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026920647"}]},"ts":"1681026920647"} 2023-04-09 07:55:20,648 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026920647"}]},"ts":"1681026920647"} 2023-04-09 07:55:20,648 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026920647"}]},"ts":"1681026920647"} 2023-04-09 07:55:20,649 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-09 07:55:20,650 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:55:20,650 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026920650"}]},"ts":"1681026920650"} 2023-04-09 07:55:20,651 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-04-09 07:55:20,654 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=410e3eed8a4051a7a5cebb43c08ccfed, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c669f14846d655d587709ca8bd4b7e00, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=351ba7e24003baba73ac0a889f713678, ASSIGN}] 2023-04-09 07:55:20,656 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=410e3eed8a4051a7a5cebb43c08ccfed, ASSIGN 2023-04-09 07:55:20,656 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=351ba7e24003baba73ac0a889f713678, ASSIGN 2023-04-09 07:55:20,656 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c669f14846d655d587709ca8bd4b7e00, ASSIGN 2023-04-09 07:55:20,656 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=410e3eed8a4051a7a5cebb43c08ccfed, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:20,656 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=351ba7e24003baba73ac0a889f713678, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:20,656 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c669f14846d655d587709ca8bd4b7e00, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:20,689 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:20,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_] to archive 2023-04-09 07:55:20,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:20,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-dc3980414a4348f6a3429b4c34391e56 2023-04-09 07:55:20,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5987d3843a9541ec9705d9216fa5d916 2023-04-09 07:55:20,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-0512b4adaf1b4014a3bf577864b0cd49 2023-04-09 07:55:20,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-8530d077b2fb44e8b1bf6dbdc609356d 2023-04-09 07:55:20,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family1/f665ed52f94549059ab4d25040d0c561_SeqId_34_ 2023-04-09 07:55:20,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_] to archive 2023-04-09 07:55:20,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:20,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-6e82ea65bd89416ba123d8deb4c583f2 2023-04-09 07:55:20,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-d251bda3f1d84e7f94675a658f2caf7e 2023-04-09 07:55:20,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-489073fa9db54eae8fe5585df2e0d914 2023-04-09 07:55:20,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c4a5a528cb424261a35132ab5f18fcd0 2023-04-09 07:55:20,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family1/fd6e9db437b140dcbe42beabb66ec312_SeqId_34_ 2023-04-09 07:55:20,809 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c669f14846d655d587709ca8bd4b7e00, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,809 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=410e3eed8a4051a7a5cebb43c08ccfed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,809 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=351ba7e24003baba73ac0a889f713678, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1681026920809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026920809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026920809"}]},"ts":"1681026920809"} 2023-04-09 07:55:20,809 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026920809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026920809"}]},"ts":"1681026920809"} 2023-04-09 07:55:20,809 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026920809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026920809"}]},"ts":"1681026920809"} 2023-04-09 07:55:20,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure c669f14846d655d587709ca8bd4b7e00, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:20,811 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure 351ba7e24003baba73ac0a889f713678, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:20,811 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 410e3eed8a4051a7a5cebb43c08ccfed, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:20,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:20,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 351ba7e24003baba73ac0a889f713678, NAME => 'ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.', STARTKEY => '2', ENDKEY => ''} 2023-04-09 07:55:20,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,966 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,967 DEBUG [StoreOpener-351ba7e24003baba73ac0a889f713678-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1 2023-04-09 07:55:20,967 DEBUG [StoreOpener-351ba7e24003baba73ac0a889f713678-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1 2023-04-09 07:55:20,967 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 351ba7e24003baba73ac0a889f713678 columnFamilyName family1 2023-04-09 07:55:20,968 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] regionserver.HStore(310): Store=351ba7e24003baba73ac0a889f713678/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,968 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,969 DEBUG [StoreOpener-351ba7e24003baba73ac0a889f713678-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2 2023-04-09 07:55:20,969 DEBUG [StoreOpener-351ba7e24003baba73ac0a889f713678-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2 2023-04-09 07:55:20,970 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 351ba7e24003baba73ac0a889f713678 columnFamilyName family2 2023-04-09 07:55:20,970 INFO [StoreOpener-351ba7e24003baba73ac0a889f713678-1] regionserver.HStore(310): Store=351ba7e24003baba73ac0a889f713678/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:20,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 351ba7e24003baba73ac0a889f713678 2023-04-09 07:55:20,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:20,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 351ba7e24003baba73ac0a889f713678; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61537107, jitterRate=-0.08302564918994904}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:20,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:20,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678., pid=110, masterSystemTime=1681026920962 2023-04-09 07:55:20,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:20,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 410e3eed8a4051a7a5cebb43c08ccfed, NAME => 'ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.', STARTKEY => '', ENDKEY => '1'} 2023-04-09 07:55:20,976 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=351ba7e24003baba73ac0a889f713678, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,977 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920976"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026920976"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026920976"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026920976"}]},"ts":"1681026920976"} 2023-04-09 07:55:20,977 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,978 DEBUG [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1 2023-04-09 07:55:20,979 DEBUG [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1 2023-04-09 07:55:20,979 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=108 2023-04-09 07:55:20,979 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 410e3eed8a4051a7a5cebb43c08ccfed columnFamilyName family1 2023-04-09 07:55:20,979 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure 351ba7e24003baba73ac0a889f713678, server=jenkins-hbase4.apache.org,34907,1681026816245 in 167 msec 2023-04-09 07:55:20,979 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] regionserver.HStore(310): Store=410e3eed8a4051a7a5cebb43c08ccfed/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,980 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=351ba7e24003baba73ac0a889f713678, ASSIGN in 325 msec 2023-04-09 07:55:20,980 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,981 DEBUG [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2 2023-04-09 07:55:20,981 DEBUG [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2 2023-04-09 07:55:20,981 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 410e3eed8a4051a7a5cebb43c08ccfed columnFamilyName family2 2023-04-09 07:55:20,982 INFO [StoreOpener-410e3eed8a4051a7a5cebb43c08ccfed-1] regionserver.HStore(310): Store=410e3eed8a4051a7a5cebb43c08ccfed/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:20,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:55:20,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:20,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 410e3eed8a4051a7a5cebb43c08ccfed; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69351083, jitterRate=0.0334116667509079}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:20,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:20,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed., pid=111, masterSystemTime=1681026920962 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:20,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c669f14846d655d587709ca8bd4b7e00, NAME => 'ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.', STARTKEY => '1', ENDKEY => '2'} 2023-04-09 07:55:20,988 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=410e3eed8a4051a7a5cebb43c08ccfed, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:20,988 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1681026920988"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026920988"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026920988"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026920988"}]},"ts":"1681026920988"} 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,989 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,990 DEBUG [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1 2023-04-09 07:55:20,990 DEBUG [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1 2023-04-09 07:55:20,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-04-09 07:55:20,990 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c669f14846d655d587709ca8bd4b7e00 columnFamilyName family1 2023-04-09 07:55:20,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 410e3eed8a4051a7a5cebb43c08ccfed, server=jenkins-hbase4.apache.org,34907,1681026816245 in 178 msec 2023-04-09 07:55:20,991 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] regionserver.HStore(310): Store=c669f14846d655d587709ca8bd4b7e00/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,991 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=410e3eed8a4051a7a5cebb43c08ccfed, ASSIGN in 336 msec 2023-04-09 07:55:20,991 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,992 DEBUG [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2 2023-04-09 07:55:20,992 DEBUG [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2 2023-04-09 07:55:20,992 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c669f14846d655d587709ca8bd4b7e00 columnFamilyName family2 2023-04-09 07:55:20,993 INFO [StoreOpener-c669f14846d655d587709ca8bd4b7e00-1] regionserver.HStore(310): Store=c669f14846d655d587709ca8bd4b7e00/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:20,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:20,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:55:20,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:20,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c669f14846d655d587709ca8bd4b7e00; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72598139, jitterRate=0.0817965716123581}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:20,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:20,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00., pid=109, masterSystemTime=1681026920962 2023-04-09 07:55:20,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:20,999 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c669f14846d655d587709ca8bd4b7e00, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:20,999 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1681026920999"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026920999"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026920999"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026920999"}]},"ts":"1681026920999"} 2023-04-09 07:55:21,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-04-09 07:55:21,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure c669f14846d655d587709ca8bd4b7e00, server=jenkins-hbase4.apache.org,34907,1681026816245 in 190 msec 2023-04-09 07:55:21,002 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-04-09 07:55:21,002 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c669f14846d655d587709ca8bd4b7e00, ASSIGN in 347 msec 2023-04-09 07:55:21,002 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:55:21,003 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026921002"}]},"ts":"1681026921002"} 2023-04-09 07:55:21,003 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-04-09 07:55:21,005 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:55:21,006 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1800 sec 2023-04-09 07:55:21,458 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/.tmp/family2/6c0a0bf543ff424f9bd8dabdf3d82ae5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/6c0a0bf543ff424f9bd8dabdf3d82ae5 2023-04-09 07:55:21,462 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 9be6d4060ae57ce72cac4673686ad32d/family2 of 9be6d4060ae57ce72cac4673686ad32d into 6c0a0bf543ff424f9bd8dabdf3d82ae5(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:21,462 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:55:21,462 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d., storeName=9be6d4060ae57ce72cac4673686ad32d/family2, priority=12, startTime=1681026918644; duration=0sec 2023-04-09 07:55:21,462 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-09 07:55:21,462 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. because compaction request was cancelled 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-09 07:55:21,463 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. because compaction request was cancelled 2023-04-09 07:55:21,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7] to archive 2023-04-09 07:55:21,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:21,509 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-048a51cc9c4e4092893240c4631a54e3 2023-04-09 07:55:21,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-5d4812eb929743ffb69372e49be65034 2023-04-09 07:55:21,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-e0a6f1d523c140308259de663aae00e6 2023-04-09 07:55:21,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/family2/testCompactionWithHFileLink=9be6d4060ae57ce72cac4673686ad32d-b33d9c23d2a24add9cd9e8dd3382c4a7 2023-04-09 07:55:21,854 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/.tmp/family2/4ebd8d6f5673439aa8e0f35dbb239667 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/4ebd8d6f5673439aa8e0f35dbb239667 2023-04-09 07:55:21,858 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 60a1e7a3b2b7470f369b184b87cd1dee/family2 of 60a1e7a3b2b7470f369b184b87cd1dee into 4ebd8d6f5673439aa8e0f35dbb239667(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:21,858 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:55:21,858 INFO [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee., storeName=60a1e7a3b2b7470f369b184b87cd1dee/family2, priority=12, startTime=1681026918587; duration=1sec 2023-04-09 07:55:21,858 DEBUG [RS:0;jenkins-hbase4:34907-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:21,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2] to archive 2023-04-09 07:55:21,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:21,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-32d1220d842340d789e60950b9a1ba3e 2023-04-09 07:55:21,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-c797a98e70fa4ba1988dbd4005f84ea9 2023-04-09 07:55:21,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-338c2bebb8d54f2880050184268fa31e 2023-04-09 07:55:21,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/family2/testCompactionWithHFileLink=60a1e7a3b2b7470f369b184b87cd1dee-9050132042ca4d8296073af7e0057ac2 2023-04-09 07:55:21,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-09 07:55:21,933 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-04-09 07:55:21,940 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:21,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:21,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:21,943 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:21,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-09 07:55:21,945 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-09 07:55:21,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:21,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-09 07:55:21,945 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-09 07:55:21,946 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:21,946 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:21,947 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:21,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:21,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:21,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:21,948 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:21,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:21,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-09 07:55:21,949 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:21,949 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:21,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-09 07:55:21,949 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-09 07:55:21,949 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:21,949 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-09 07:55:21,949 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:21,949 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:21,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:21,950 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. started... 2023-04-09 07:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. started... 2023-04-09 07:55:21,950 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:21,952 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. started... 2023-04-09 07:55:21,952 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 410e3eed8a4051a7a5cebb43c08ccfed 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:21,952 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c669f14846d655d587709ca8bd4b7e00 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:21,953 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 351ba7e24003baba73ac0a889f713678 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:22,398 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409fc8e156c43f448aba20bfa1e1bfba498 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409fc8e156c43f448aba20bfa1e1bfba498 2023-04-09 07:55:22,413 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409d0cebcad88b54619b54a2425e0cbbb0b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230409d0cebcad88b54619b54a2425e0cbbb0b 2023-04-09 07:55:22,414 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040974a4402cee9441b99e86f6ec7fb9299b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023040974a4402cee9441b99e86f6ec7fb9299b 2023-04-09 07:55:22,428 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/1af4123c4e0e4551a9301e7247338392 2023-04-09 07:55:22,803 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/820b51ab68994f8885b878e8451c2a8c 2023-04-09 07:55:22,832 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/4c95f8b114c944b2bdad5c61d9c6b842 2023-04-09 07:55:22,856 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409dc2f7f7502914a66b5c9ec1a15f75d62 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230409dc2f7f7502914a66b5c9ec1a15f75d62 2023-04-09 07:55:23,246 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409f5c615713a3143ac91dadca8c7f0683c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230409f5c615713a3143ac91dadca8c7f0683c 2023-04-09 07:55:23,262 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409a627f642feb44ca5a5d777b7ce551ded to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230409a627f642feb44ca5a5d777b7ce551ded 2023-04-09 07:55:23,263 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/bd3d1786dc47404a953c1d4fa6f1e718 2023-04-09 07:55:23,276 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/1af4123c4e0e4551a9301e7247338392 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392 2023-04-09 07:55:23,277 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/856508a7ca1f493db0fbcd880c2e9271 2023-04-09 07:55:23,281 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:55:23,282 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/bd3d1786dc47404a953c1d4fa6f1e718 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718 2023-04-09 07:55:23,282 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/4c95f8b114c944b2bdad5c61d9c6b842 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842 2023-04-09 07:55:23,286 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:55:23,286 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:23,287 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/856508a7ca1f493db0fbcd880c2e9271 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271 2023-04-09 07:55:23,287 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 410e3eed8a4051a7a5cebb43c08ccfed in 1335ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:23,287 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-04-09 07:55:23,288 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:23,288 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:23,288 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:23,291 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:23,292 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 351ba7e24003baba73ac0a889f713678 in 1339ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:23,293 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:23,293 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:23,293 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:23,652 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/532fefb49d1a443fbba1b7dc3219e3d9 2023-04-09 07:55:23,657 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/820b51ab68994f8885b878e8451c2a8c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c 2023-04-09 07:55:23,662 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c, entries=4, sequenceid=5, filesize=5.4 K 2023-04-09 07:55:23,663 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/532fefb49d1a443fbba1b7dc3219e3d9 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9 2023-04-09 07:55:23,667 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:23,668 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c669f14846d655d587709ca8bd4b7e00 in 1716ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:23,668 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:23,668 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-09 07:55:23,668 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-09 07:55:23,670 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,670 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:23,671 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:23,671 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:23,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:23,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-09 07:55:23,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19f6376[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:23,672 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-09 07:55:23,672 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,673 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,674 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-09 07:55:23,674 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-09 07:55:23,674 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:23,674 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,674 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:23,675 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-09 07:55:23,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:23,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:23,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:23,675 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-09 07:55:23,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:23,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-09 07:55:23,677 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-09 07:55:23,677 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:23,677 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:23,678 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-09 07:55:23,678 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:23,679 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,679 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:23,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:23,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,679 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:23,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:23,679 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:23,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:23,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:23,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,681 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:23,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:23,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:23,684 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-09 07:55:23,684 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:23,684 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:23,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:23,684 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:23,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:23,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:23,785 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:23,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:23,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:23,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:23,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:23,791 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:23,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:23,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-09 07:55:23,793 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-09 07:55:23,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:23,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-09 07:55:23,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-09 07:55:23,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,793 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:23,794 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:23,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:23,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:23,795 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:23,795 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:23,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:23,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:23,796 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-09 07:55:23,796 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-09 07:55:23,796 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-09 07:55:23,796 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-09 07:55:23,796 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:23,796 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-09 07:55:23,797 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:23,797 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. started... 2023-04-09 07:55:23,797 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 410e3eed8a4051a7a5cebb43c08ccfed 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:23,800 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:23,800 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. started... 2023-04-09 07:55:23,800 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 351ba7e24003baba73ac0a889f713678 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:23,808 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:23,808 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. started... 2023-04-09 07:55:23,808 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c669f14846d655d587709ca8bd4b7e00 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:23,850 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304095b4ca75879454bcba2dddb68048ae208 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304095b4ca75879454bcba2dddb68048ae208 2023-04-09 07:55:23,865 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/0520d0678f0d48788488f73eefcb84b1 2023-04-09 07:55:24,243 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040978a027bf1a2b4cd79037df82f244349e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023040978a027bf1a2b4cd79037df82f244349e 2023-04-09 07:55:24,248 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/0d21891832644ca0b5149130c4c7104f 2023-04-09 07:55:24,262 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409d87dbd8c6d0d4489b15092c44045eee6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409d87dbd8c6d0d4489b15092c44045eee6 2023-04-09 07:55:24,283 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409b0ba7baa47b54e71948ed932a5657145 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230409b0ba7baa47b54e71948ed932a5657145 2023-04-09 07:55:24,661 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409d00ac0283f69487f8d1e12d1d0e605c2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230409d00ac0283f69487f8d1e12d1d0e605c2 2023-04-09 07:55:24,667 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/4da45201b05f4dbb97f98e66c6142135 2023-04-09 07:55:24,668 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/7ed842a55a8043fcb14cd9175342d775 2023-04-09 07:55:24,672 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/0d21891832644ca0b5149130c4c7104f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f 2023-04-09 07:55:24,678 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:55:24,678 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/7ed842a55a8043fcb14cd9175342d775 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775 2023-04-09 07:55:24,684 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:24,685 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 410e3eed8a4051a7a5cebb43c08ccfed in 888ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:24,685 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:24,685 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:24,685 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:24,693 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/4e3b43f9bef747acb11375c01d3a97d5 2023-04-09 07:55:24,698 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/0520d0678f0d48788488f73eefcb84b1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1 2023-04-09 07:55:24,700 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409e8007f071643479ebb7be8b9b13585d4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230409e8007f071643479ebb7be8b9b13585d4 2023-04-09 07:55:24,706 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:55:24,707 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/4e3b43f9bef747acb11375c01d3a97d5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5 2023-04-09 07:55:24,711 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:24,712 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 351ba7e24003baba73ac0a889f713678 in 912ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:24,712 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:24,712 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:24,712 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:25,125 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/1c02737c2e8746f68bfb1c50fbf1ab85 2023-04-09 07:55:25,129 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/4da45201b05f4dbb97f98e66c6142135 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135 2023-04-09 07:55:25,132 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135, entries=4, sequenceid=9, filesize=5.4 K 2023-04-09 07:55:25,133 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/1c02737c2e8746f68bfb1c50fbf1ab85 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85 2023-04-09 07:55:25,136 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:25,137 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c669f14846d655d587709ca8bd4b7e00 in 1329ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:25,137 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:25,137 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:25,137 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-09 07:55:25,138 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-09 07:55:25,139 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:25,144 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:25,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:25,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:25,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-09 07:55:25,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a39ddf2[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:25,146 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-09 07:55:25,146 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,147 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,147 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-09 07:55:25,147 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-09 07:55:25,147 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:25,147 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,147 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-09 07:55:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:25,156 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-09 07:55:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:25,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:25,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-09 07:55:25,158 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-09 07:55:25,158 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:25,158 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:25,159 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-09 07:55:25,159 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:25,160 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,160 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,160 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:25,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:25,161 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:25,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:25,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:25,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:25,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:25,164 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-09 07:55:25,164 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:25,164 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:25,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:25,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:25,164 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,165 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:25,165 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,165 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:25,165 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,265 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:25,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:25,268 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:25,268 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:25,268 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:25,271 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:25,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:25,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-09 07:55:25,273 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-09 07:55:25,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:25,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-09 07:55:25,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-09 07:55:25,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,273 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:25,276 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:25,276 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:25,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:25,276 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:25,276 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:25,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:25,277 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:25,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:25,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:25,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-09 07:55:25,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-09 07:55:25,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-09 07:55:25,277 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-09 07:55:25,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:25,278 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-09 07:55:25,278 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:25,278 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. started... 2023-04-09 07:55:25,278 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 410e3eed8a4051a7a5cebb43c08ccfed 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:25,279 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:25,279 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. started... 2023-04-09 07:55:25,279 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 351ba7e24003baba73ac0a889f713678 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:25,285 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:25,285 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:25,285 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:25,285 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. started... 2023-04-09 07:55:25,285 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c669f14846d655d587709ca8bd4b7e00 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:25,318 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304091528fa853e44484fb30f3e04ce159501 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202304091528fa853e44484fb30f3e04ce159501 2023-04-09 07:55:25,319 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409e2efc3599f4e4a3c820dfdc488f39229 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230409e2efc3599f4e4a3c820dfdc488f39229 2023-04-09 07:55:25,332 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/5db68b89e733476e97ad807d59c5f1c7 2023-04-09 07:55:25,333 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/e848c73f66ae4dc285d204047320b22b 2023-04-09 07:55:25,715 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304098c3042e1287c48519f4d839c283b587f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304098c3042e1287c48519f4d839c283b587f 2023-04-09 07:55:25,721 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/0e6f177466904a168bf1839f22a419a5 2023-04-09 07:55:25,773 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304090c7c86771fdd4a9ea0d3bdd44f5295fa to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202304090c7c86771fdd4a9ea0d3bdd44f5295fa 2023-04-09 07:55:25,773 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409143d8ed4da6e47e3a04f07393dd7b025 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230409143d8ed4da6e47e3a04f07393dd7b025 2023-04-09 07:55:26,134 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409c1de376dc0c04e968eebadf3e1a37085 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230409c1de376dc0c04e968eebadf3e1a37085 2023-04-09 07:55:26,183 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/842260715c0f4729b09c8e44375750e6 2023-04-09 07:55:26,184 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/bee54eb2fb694525aaabd74fed2bb965 2023-04-09 07:55:26,187 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/5db68b89e733476e97ad807d59c5f1c7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7 2023-04-09 07:55:26,188 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/e848c73f66ae4dc285d204047320b22b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b 2023-04-09 07:55:26,191 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:26,192 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:26,192 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/842260715c0f4729b09c8e44375750e6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6 2023-04-09 07:55:26,192 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/bee54eb2fb694525aaabd74fed2bb965 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965 2023-04-09 07:55:26,196 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:26,196 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:26,197 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 351ba7e24003baba73ac0a889f713678 in 918ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:26,197 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c669f14846d655d587709ca8bd4b7e00 in 912ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:26,197 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:26,197 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:26,197 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:26,197 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:26,197 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:26,198 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:26,466 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:26,539 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/847b2ab6928848c781b4d81c9bb6b02a 2023-04-09 07:55:26,544 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/0e6f177466904a168bf1839f22a419a5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5 2023-04-09 07:55:26,551 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5, entries=4, sequenceid=13, filesize=5.4 K 2023-04-09 07:55:26,552 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/847b2ab6928848c781b4d81c9bb6b02a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a 2023-04-09 07:55:26,556 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:26,557 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 410e3eed8a4051a7a5cebb43c08ccfed in 1279ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:26,557 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:26,557 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-09 07:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-09 07:55:26,559 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:26,559 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:26,560 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,560 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:26,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:26,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:26,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-09 07:55:26,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a3f6af8[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:26,561 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-09 07:55:26,561 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,562 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,562 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-09 07:55:26,563 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-09 07:55:26,563 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:26,563 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,563 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:26,564 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:26,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-09 07:55:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:26,564 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-09 07:55:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:26,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:26,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:26,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:26,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-09 07:55:26,568 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-09 07:55:26,568 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:26,568 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:26,568 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-09 07:55:26,568 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:26,571 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,571 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:26,571 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:26,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:26,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:26,572 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:26,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,574 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:26,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:26,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:26,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-09 07:55:26,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:26,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:26,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:26,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:26,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:26,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:26,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:26,578 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,678 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:26,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:26,680 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:26,680 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:26,680 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:26,683 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:26,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:26,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-09 07:55:26,687 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-09 07:55:26,687 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:26,687 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-09 07:55:26,687 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-09 07:55:26,687 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,687 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:26,689 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:26,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:26,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:26,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:26,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:26,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:26,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-09 07:55:26,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-09 07:55:26,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-09 07:55:26,690 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-09 07:55:26,691 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:26,691 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-09 07:55:26,691 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:26,691 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. started... 2023-04-09 07:55:26,691 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:26,691 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 410e3eed8a4051a7a5cebb43c08ccfed 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:26,691 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. started... 2023-04-09 07:55:26,691 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 351ba7e24003baba73ac0a889f713678 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:26,692 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:26,692 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. started... 2023-04-09 07:55:26,692 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c669f14846d655d587709ca8bd4b7e00 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:26,727 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ce03f10af3e34aba9810a551036b0703 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce03f10af3e34aba9810a551036b0703 2023-04-09 07:55:26,729 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304097971e19005af45fcbe9a1e0df34b700a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202304097971e19005af45fcbe9a1e0df34b700a 2023-04-09 07:55:27,121 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304096afa7e6cb51d4a35bb67a27593bb57c1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202304096afa7e6cb51d4a35bb67a27593bb57c1 2023-04-09 07:55:27,142 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/a80cccfdc38a48ab8936f1804ed60d6b 2023-04-09 07:55:27,142 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/db680811ea104adc961da3a56ab92d51 2023-04-09 07:55:27,526 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/767c0e3caac74e18a3e5e154b3f3f11c 2023-04-09 07:55:27,561 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304099ad7581363cf44208f0045e042882afc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202304099ad7581363cf44208f0045e042882afc 2023-04-09 07:55:27,561 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304092622003d69704aba9ccc4bb9e5f3e848 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202304092622003d69704aba9ccc4bb9e5f3e848 2023-04-09 07:55:27,940 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304091dc187ea859346aa84a5693d885666fa to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202304091dc187ea859346aa84a5693d885666fa 2023-04-09 07:55:27,966 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/0a4c779c2f5044129367ef47d8db8dcd 2023-04-09 07:55:27,966 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/61b11c3ead0849c9adeb8c0efd6eb377 2023-04-09 07:55:27,971 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/a80cccfdc38a48ab8936f1804ed60d6b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b 2023-04-09 07:55:27,972 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/db680811ea104adc961da3a56ab92d51 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51 2023-04-09 07:55:27,976 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:27,976 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:27,977 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/0a4c779c2f5044129367ef47d8db8dcd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd 2023-04-09 07:55:27,979 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/61b11c3ead0849c9adeb8c0efd6eb377 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377 2023-04-09 07:55:27,985 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:27,985 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:27,986 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 410e3eed8a4051a7a5cebb43c08ccfed in 1295ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:27,986 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 351ba7e24003baba73ac0a889f713678 in 1295ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:27,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:27,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:27,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:27,986 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:27,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:27,986 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:28,346 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/35e791e10d9c4f2fb8d8ad4e058a4086 2023-04-09 07:55:28,353 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/767c0e3caac74e18a3e5e154b3f3f11c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c 2023-04-09 07:55:28,357 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c, entries=4, sequenceid=17, filesize=5.4 K 2023-04-09 07:55:28,358 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/35e791e10d9c4f2fb8d8ad4e058a4086 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086 2023-04-09 07:55:28,362 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:28,362 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c669f14846d655d587709ca8bd4b7e00 in 1670ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:28,362 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:28,362 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:28,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:28,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:28,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-09 07:55:28,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-09 07:55:28,364 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,364 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:28,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:28,365 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,365 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:28,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:28,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:28,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:28,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-09 07:55:28,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35a97538[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:28,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-09 07:55:28,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,368 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-09 07:55:28,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-09 07:55:28,368 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:28,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,369 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:28,370 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-09 07:55:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:28,370 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-09 07:55:28,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:28,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-09 07:55:28,372 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-09 07:55:28,373 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:28,373 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:28,373 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-09 07:55:28,373 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:28,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:28,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,376 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,376 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:28,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,376 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:28,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:28,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:28,377 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:28,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,380 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:28,380 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:28,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:28,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:28,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:28,381 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,381 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-09 07:55:28,381 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:28,381 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:28,381 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:28,381 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:28,381 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:28,382 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:28,382 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,382 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,382 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:28,382 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,482 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:28,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:28,491 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:28,500 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:28,509 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:28,510 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:28,537 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:28,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:28,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-04-09 07:55:28,538 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-04-09 07:55:28,539 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:28,539 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-04-09 07:55:28,539 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-04-09 07:55:28,539 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,539 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:28,540 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:28,540 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:28,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:28,541 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:28,541 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:28,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:28,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:28,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:28,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:28,542 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-04-09 07:55:28,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-04-09 07:55:28,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-04-09 07:55:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-04-09 07:55:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. started... 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:28,548 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 410e3eed8a4051a7a5cebb43c08ccfed 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. started... 2023-04-09 07:55:28,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. started... 2023-04-09 07:55:28,548 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:28,549 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing c669f14846d655d587709ca8bd4b7e00 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:28,549 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 351ba7e24003baba73ac0a889f713678 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:28,597 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/296eb9525a0f4074b5dcdec69c58489d 2023-04-09 07:55:28,598 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/84e369ae9c92487782cd65fdff958320 2023-04-09 07:55:28,602 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 296eb9525a0f4074b5dcdec69c58489d 2023-04-09 07:55:28,602 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84e369ae9c92487782cd65fdff958320 2023-04-09 07:55:28,991 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/e48a4999ca274c35bf4bf36829570c00 2023-04-09 07:55:28,995 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e48a4999ca274c35bf4bf36829570c00 2023-04-09 07:55:29,029 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/1e0faca8244446f29492ef23a71b1dae 2023-04-09 07:55:29,032 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e0faca8244446f29492ef23a71b1dae 2023-04-09 07:55:29,033 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/84e369ae9c92487782cd65fdff958320 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320 2023-04-09 07:55:29,033 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/a5b36ce3f6054d4a88101cb0dc1fc65e 2023-04-09 07:55:29,036 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5b36ce3f6054d4a88101cb0dc1fc65e 2023-04-09 07:55:29,037 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84e369ae9c92487782cd65fdff958320 2023-04-09 07:55:29,037 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320, entries=3, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,037 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/296eb9525a0f4074b5dcdec69c58489d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d 2023-04-09 07:55:29,038 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/1e0faca8244446f29492ef23a71b1dae as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae 2023-04-09 07:55:29,041 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 296eb9525a0f4074b5dcdec69c58489d 2023-04-09 07:55:29,041 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d, entries=3, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,042 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1e0faca8244446f29492ef23a71b1dae 2023-04-09 07:55:29,042 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,042 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/a5b36ce3f6054d4a88101cb0dc1fc65e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e 2023-04-09 07:55:29,043 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 351ba7e24003baba73ac0a889f713678 in 494ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:29,043 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:29,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:29,046 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a5b36ce3f6054d4a88101cb0dc1fc65e 2023-04-09 07:55:29,046 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,047 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 410e3eed8a4051a7a5cebb43c08ccfed in 499ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:29,047 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:29,047 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:29,047 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:29,819 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/e52a02ff2d78447689f1aaa92ce5b3be 2023-04-09 07:55:29,823 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e52a02ff2d78447689f1aaa92ce5b3be 2023-04-09 07:55:29,824 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/e48a4999ca274c35bf4bf36829570c00 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00 2023-04-09 07:55:29,828 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e48a4999ca274c35bf4bf36829570c00 2023-04-09 07:55:29,828 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00, entries=3, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,829 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/e52a02ff2d78447689f1aaa92ce5b3be as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be 2023-04-09 07:55:29,833 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e52a02ff2d78447689f1aaa92ce5b3be 2023-04-09 07:55:29,833 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:29,834 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c669f14846d655d587709ca8bd4b7e00 in 1285ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:29,834 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:29,834 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:29,834 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:29,834 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-04-09 07:55:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-04-09 07:55:29,836 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,836 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:29,837 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,837 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:29,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-04-09 07:55:29,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fd72428[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:29,838 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-04-09 07:55:29,838 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,840 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-04-09 07:55:29,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-04-09 07:55:29,840 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,840 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:29,840 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:29,842 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-04-09 07:55:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:29,842 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-04-09 07:55:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:29,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-04-09 07:55:29,844 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-04-09 07:55:29,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:29,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:29,844 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-04-09 07:55:29,844 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:29,848 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,848 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:29,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:29,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,848 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:29,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:29,849 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,850 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:29,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-04-09 07:55:29,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,853 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,853 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:29,853 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:29,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:29,853 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:29,853 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-04-09 07:55:29,854 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:29,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:29,854 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-04-09 07:55:29,854 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:29,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:29,854 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:29,954 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:29,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:29,956 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:29,956 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 410e3eed8a4051a7a5cebb43c08ccfed/family1 is initiating major compaction (all files) 2023-04-09 07:55:29,956 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:29,956 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:29,956 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 410e3eed8a4051a7a5cebb43c08ccfed/family1 in ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:29,957 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp, totalSize=26.6 K 2023-04-09 07:55:29,957 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 1af4123c4e0e4551a9301e7247338392, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921940 2023-04-09 07:55:29,958 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 0d21891832644ca0b5149130c4c7104f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:29,958 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 0e6f177466904a168bf1839f22a419a5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925267 2023-04-09 07:55:29,958 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting db680811ea104adc961da3a56ab92d51, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:29,959 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 296eb9525a0f4074b5dcdec69c58489d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:29,971 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 410e3eed8a4051a7a5cebb43c08ccfed#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:30,380 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409760551653b1846adb4272a770c8d7718_del 2023-04-09 07:55:30,380 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409760551653b1846adb4272a770c8d7718_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230409760551653b1846adb4272a770c8d7718_del 2023-04-09 07:55:31,195 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 398de871c4724da18ca30050d8e8a566 2023-04-09 07:55:31,196 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family1/398de871c4724da18ca30050d8e8a566 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/398de871c4724da18ca30050d8e8a566 2023-04-09 07:55:31,201 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 398de871c4724da18ca30050d8e8a566 2023-04-09 07:55:31,202 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 410e3eed8a4051a7a5cebb43c08ccfed/family1 of 410e3eed8a4051a7a5cebb43c08ccfed into 398de871c4724da18ca30050d8e8a566(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:31,202 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:31,202 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:31,203 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 410e3eed8a4051a7a5cebb43c08ccfed/family2 is initiating major compaction (all files) 2023-04-09 07:55:31,203 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:31,203 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:31,203 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 410e3eed8a4051a7a5cebb43c08ccfed/family2 in ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:55:31,203 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp, totalSize=25.3 K 2023-04-09 07:55:31,203 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting bd3d1786dc47404a953c1d4fa6f1e718, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921940 2023-04-09 07:55:31,204 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 7ed842a55a8043fcb14cd9175342d775, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:31,204 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 847b2ab6928848c781b4d81c9bb6b02a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925267 2023-04-09 07:55:31,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d] to archive 2023-04-09 07:55:31,212 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 0a4c779c2f5044129367ef47d8db8dcd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:31,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:31,212 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting a5b36ce3f6054d4a88101cb0dc1fc65e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:31,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/1af4123c4e0e4551a9301e7247338392 2023-04-09 07:55:31,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0d21891832644ca0b5149130c4c7104f 2023-04-09 07:55:31,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0e6f177466904a168bf1839f22a419a5 2023-04-09 07:55:31,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/db680811ea104adc961da3a56ab92d51 2023-04-09 07:55:31,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/296eb9525a0f4074b5dcdec69c58489d 2023-04-09 07:55:31,227 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 410e3eed8a4051a7a5cebb43c08ccfed#family2#compaction#150 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:31,237 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202304091f62ce0967b04c06aeb24069bb51743f_del 2023-04-09 07:55:31,237 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304091f62ce0967b04c06aeb24069bb51743f_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202304091f62ce0967b04c06aeb24069bb51743f_del 2023-04-09 07:55:31,254 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a77b9640ff7a4de791661608cb84385d 2023-04-09 07:55:31,255 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/.tmp/family2/a77b9640ff7a4de791661608cb84385d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a77b9640ff7a4de791661608cb84385d 2023-04-09 07:55:31,259 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a77b9640ff7a4de791661608cb84385d 2023-04-09 07:55:31,260 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 410e3eed8a4051a7a5cebb43c08ccfed/family2 of 410e3eed8a4051a7a5cebb43c08ccfed into a77b9640ff7a4de791661608cb84385d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:31,260 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:31,261 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:31,261 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 351ba7e24003baba73ac0a889f713678/family1 is initiating major compaction (all files) 2023-04-09 07:55:31,261 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:31,261 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:31,262 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 351ba7e24003baba73ac0a889f713678/family1 in ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:31,262 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp, totalSize=26.6 K 2023-04-09 07:55:31,264 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 4c95f8b114c944b2bdad5c61d9c6b842, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921941 2023-04-09 07:55:31,264 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 0520d0678f0d48788488f73eefcb84b1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:31,265 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 5db68b89e733476e97ad807d59c5f1c7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925268 2023-04-09 07:55:31,265 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting a80cccfdc38a48ab8936f1804ed60d6b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:31,266 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 84e369ae9c92487782cd65fdff958320, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:31,285 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 351ba7e24003baba73ac0a889f713678#family1#compaction#151 average throughput is 0.28 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:31,306 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202304098c4179dd92e64b1d84589ea9ad854b4e_del 2023-04-09 07:55:31,306 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304098c4179dd92e64b1d84589ea9ad854b4e_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304098c4179dd92e64b1d84589ea9ad854b4e_del 2023-04-09 07:55:31,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e] to archive 2023-04-09 07:55:31,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:31,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/bd3d1786dc47404a953c1d4fa6f1e718 2023-04-09 07:55:31,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/7ed842a55a8043fcb14cd9175342d775 2023-04-09 07:55:31,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/847b2ab6928848c781b4d81c9bb6b02a 2023-04-09 07:55:31,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/0a4c779c2f5044129367ef47d8db8dcd 2023-04-09 07:55:31,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family2/a5b36ce3f6054d4a88101cb0dc1fc65e 2023-04-09 07:55:32,136 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e147897cd494201a1a78f1f2aa1f90e 2023-04-09 07:55:32,137 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family1/7e147897cd494201a1a78f1f2aa1f90e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/7e147897cd494201a1a78f1f2aa1f90e 2023-04-09 07:55:32,142 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7e147897cd494201a1a78f1f2aa1f90e 2023-04-09 07:55:32,143 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 351ba7e24003baba73ac0a889f713678/family1 of 351ba7e24003baba73ac0a889f713678 into 7e147897cd494201a1a78f1f2aa1f90e(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:32,143 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:32,144 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:32,144 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): 351ba7e24003baba73ac0a889f713678/family2 is initiating major compaction (all files) 2023-04-09 07:55:32,144 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:32,144 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:32,149 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of 351ba7e24003baba73ac0a889f713678/family2 in ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:55:32,149 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp, totalSize=25.3 K 2023-04-09 07:55:32,150 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 856508a7ca1f493db0fbcd880c2e9271, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921941 2023-04-09 07:55:32,150 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 4e3b43f9bef747acb11375c01d3a97d5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:32,151 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 842260715c0f4729b09c8e44375750e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925268 2023-04-09 07:55:32,151 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 61b11c3ead0849c9adeb8c0efd6eb377, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:32,151 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 1e0faca8244446f29492ef23a71b1dae, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:32,165 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): 351ba7e24003baba73ac0a889f713678#family2#compaction#152 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:32,178 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230409fa51bca9d24947eca9c37f74dfcc8487_del 2023-04-09 07:55:32,178 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409fa51bca9d24947eca9c37f74dfcc8487_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230409fa51bca9d24947eca9c37f74dfcc8487_del 2023-04-09 07:55:32,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320] to archive 2023-04-09 07:55:32,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:32,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/4c95f8b114c944b2bdad5c61d9c6b842 2023-04-09 07:55:32,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/0520d0678f0d48788488f73eefcb84b1 2023-04-09 07:55:32,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/5db68b89e733476e97ad807d59c5f1c7 2023-04-09 07:55:32,212 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/a80cccfdc38a48ab8936f1804ed60d6b 2023-04-09 07:55:32,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/84e369ae9c92487782cd65fdff958320 2023-04-09 07:55:32,596 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c5a37a9df7b450f823f53c1173eaaf4 2023-04-09 07:55:32,597 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/.tmp/family2/7c5a37a9df7b450f823f53c1173eaaf4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/7c5a37a9df7b450f823f53c1173eaaf4 2023-04-09 07:55:32,600 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7c5a37a9df7b450f823f53c1173eaaf4 2023-04-09 07:55:32,602 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 351ba7e24003baba73ac0a889f713678/family2 of 351ba7e24003baba73ac0a889f713678 into 7c5a37a9df7b450f823f53c1173eaaf4(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:32,602 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:32,602 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:32,602 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): c669f14846d655d587709ca8bd4b7e00/family1 is initiating major compaction (all files) 2023-04-09 07:55:32,602 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:32,602 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:32,602 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of c669f14846d655d587709ca8bd4b7e00/family1 in ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:32,603 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp, totalSize=26.6 K 2023-04-09 07:55:32,603 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 820b51ab68994f8885b878e8451c2a8c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921941 2023-04-09 07:55:32,603 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 4da45201b05f4dbb97f98e66c6142135, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:32,604 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting e848c73f66ae4dc285d204047320b22b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925268 2023-04-09 07:55:32,604 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 767c0e3caac74e18a3e5e154b3f3f11c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:32,604 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting e48a4999ca274c35bf4bf36829570c00, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:32,605 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae] to archive 2023-04-09 07:55:32,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:32,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/856508a7ca1f493db0fbcd880c2e9271 2023-04-09 07:55:32,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/4e3b43f9bef747acb11375c01d3a97d5 2023-04-09 07:55:32,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/842260715c0f4729b09c8e44375750e6 2023-04-09 07:55:32,612 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/61b11c3ead0849c9adeb8c0efd6eb377 2023-04-09 07:55:32,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family2/1e0faca8244446f29492ef23a71b1dae 2023-04-09 07:55:32,616 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): c669f14846d655d587709ca8bd4b7e00#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:32,633 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409c97d6b17545d47598eec489d0690ae90_del 2023-04-09 07:55:32,633 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409c97d6b17545d47598eec489d0690ae90_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409c97d6b17545d47598eec489d0690ae90_del 2023-04-09 07:55:33,054 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 041f2e47e0ed4ed29cfdb0913777e70f 2023-04-09 07:55:33,054 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family1/041f2e47e0ed4ed29cfdb0913777e70f as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/041f2e47e0ed4ed29cfdb0913777e70f 2023-04-09 07:55:33,066 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 041f2e47e0ed4ed29cfdb0913777e70f 2023-04-09 07:55:33,070 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c669f14846d655d587709ca8bd4b7e00/family1 of c669f14846d655d587709ca8bd4b7e00 into 041f2e47e0ed4ed29cfdb0913777e70f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:33,070 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:33,070 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:33,070 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): c669f14846d655d587709ca8bd4b7e00/family2 is initiating major compaction (all files) 2023-04-09 07:55:33,070 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:33,070 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:33,070 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of c669f14846d655d587709ca8bd4b7e00/family2 in ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:55:33,071 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp, totalSize=25.3 K 2023-04-09 07:55:33,076 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 532fefb49d1a443fbba1b7dc3219e3d9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026921941 2023-04-09 07:55:33,076 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 1c02737c2e8746f68bfb1c50fbf1ab85, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026923788 2023-04-09 07:55:33,077 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting bee54eb2fb694525aaabd74fed2bb965, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026925268 2023-04-09 07:55:33,077 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 35e791e10d9c4f2fb8d8ad4e058a4086, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026926680 2023-04-09 07:55:33,077 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting e52a02ff2d78447689f1aaa92ce5b3be, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:33,094 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): c669f14846d655d587709ca8bd4b7e00#family2#compaction#154 average throughput is 0.20 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:33,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00] to archive 2023-04-09 07:55:33,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:33,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/820b51ab68994f8885b878e8451c2a8c 2023-04-09 07:55:33,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/4da45201b05f4dbb97f98e66c6142135 2023-04-09 07:55:33,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e848c73f66ae4dc285d204047320b22b 2023-04-09 07:55:33,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/767c0e3caac74e18a3e5e154b3f3f11c 2023-04-09 07:55:33,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/e48a4999ca274c35bf4bf36829570c00 2023-04-09 07:55:33,506 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409f52c838ff2cf43f8887200458f94fb8e_del 2023-04-09 07:55:33,507 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409f52c838ff2cf43f8887200458f94fb8e_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230409f52c838ff2cf43f8887200458f94fb8e_del 2023-04-09 07:55:34,325 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6457f4cf85d4441198cfa426c0689cbd 2023-04-09 07:55:34,326 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/.tmp/family2/6457f4cf85d4441198cfa426c0689cbd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/6457f4cf85d4441198cfa426c0689cbd 2023-04-09 07:55:34,331 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6457f4cf85d4441198cfa426c0689cbd 2023-04-09 07:55:34,332 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c669f14846d655d587709ca8bd4b7e00/family2 of c669f14846d655d587709ca8bd4b7e00 into 6457f4cf85d4441198cfa426c0689cbd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:34,332 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:34,336 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:34,339 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:34,343 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:34,343 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:34,349 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:55:34,362 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-04-09 07:55:34,365 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409760551653b1846adb4272a770c8d7718_del 2023-04-09 07:55:34,368 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409c97d6b17545d47598eec489d0690ae90_del 2023-04-09 07:55:34,371 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202304098c4179dd92e64b1d84589ea9ad854b4e_del 2023-04-09 07:55:34,371 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:34,372 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(139): Connect 0x3894c415 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:34,376 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57b6b55c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:34,378 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:34,378 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:34,378 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:34,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be] to archive 2023-04-09 07:55:34,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:34,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/532fefb49d1a443fbba1b7dc3219e3d9 2023-04-09 07:55:34,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/1c02737c2e8746f68bfb1c50fbf1ab85 2023-04-09 07:55:34,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/bee54eb2fb694525aaabd74fed2bb965 2023-04-09 07:55:34,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/35e791e10d9c4f2fb8d8ad4e058a4086 2023-04-09 07:55:34,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family2/e52a02ff2d78447689f1aaa92ce5b3be 2023-04-09 07:55:34,817 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304091208dc271ce6464b84278f501215d1cd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202304091208dc271ce6464b84278f501215d1cd 2023-04-09 07:55:34,820 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:34,839 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:34,848 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38764, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:34,858 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:55:34,861 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41692, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:55:34,888 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230409/family1/2a64779ed53a4ed29d902e33fba2d37e first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-04-09 07:55:34,889 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230409/family1/2a64779ed53a4ed29d902e33fba2d37e}] 2023-04-09 07:55:34,894 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230409/family1/2a64779ed53a4ed29d902e33fba2d37e for inclusion in 351ba7e24003baba73ac0a889f713678/family1 2023-04-09 07:55:34,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-04-09 07:55:34,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-09 07:55:34,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:55:34,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230409/family1/2a64779ed53a4ed29d902e33fba2d37e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__pp744c99ss63jdjt30rgira44lleb8t4e5quhleace92h23vifbktbo3n35i4pdl/family1/2a64779ed53a4ed29d902e33fba2d37e 2023-04-09 07:55:34,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__pp744c99ss63jdjt30rgira44lleb8t4e5quhleace92h23vifbktbo3n35i4pdl/family1/2a64779ed53a4ed29d902e33fba2d37e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/ce8c928363c0435895c17b0bd37fc43d_SeqId_28_ 2023-04-09 07:55:34,900 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__pp744c99ss63jdjt30rgira44lleb8t4e5quhleace92h23vifbktbo3n35i4pdl/family1/2a64779ed53a4ed29d902e33fba2d37e into 351ba7e24003baba73ac0a889f713678/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/ce8c928363c0435895c17b0bd37fc43d_SeqId_28_ - updating store file list. 2023-04-09 07:55:34,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/ce8c928363c0435895c17b0bd37fc43d_SeqId_28_ into 351ba7e24003baba73ac0a889f713678/family1 2023-04-09 07:55:34,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__pp744c99ss63jdjt30rgira44lleb8t4e5quhleace92h23vifbktbo3n35i4pdl/family1/2a64779ed53a4ed29d902e33fba2d37e into 351ba7e24003baba73ac0a889f713678/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/family1/ce8c928363c0435895c17b0bd37fc43d_SeqId_28_) 2023-04-09 07:55:34,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__pp744c99ss63jdjt30rgira44lleb8t4e5quhleace92h23vifbktbo3n35i4pdl/family1/2a64779ed53a4ed29d902e33fba2d37e 2023-04-09 07:55:34,906 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:34,907 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:34,909 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230409d0cebcad88b54619b54a2425e0cbbb0b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230409d0cebcad88b54619b54a2425e0cbbb0b 2023-04-09 07:55:34,909 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-09 07:55:34,909 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230409' 2023-04-09 07:55:34,909 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:35,222 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409822f009cb6824c84b02aab5be568bb2e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409822f009cb6824c84b02aab5be568bb2e 2023-04-09 07:55:35,223 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:35,238 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409d8e4c84b52df4b21a658daaff01f1ede to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230409d8e4c84b52df4b21a658daaff01f1ede 2023-04-09 07:55:35,240 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:35,250 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230409/family1/6ef59db3da3647d980a4cee3ea13441b first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-04-09 07:55:35,251 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230409/family1/6ef59db3da3647d980a4cee3ea13441b}] 2023-04-09 07:55:35,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230409/family1/6ef59db3da3647d980a4cee3ea13441b for inclusion in c669f14846d655d587709ca8bd4b7e00/family1 2023-04-09 07:55:35,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-04-09 07:55:35,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-09 07:55:35,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:55:35,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230409/family1/6ef59db3da3647d980a4cee3ea13441b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__tuv15igsvnpb43q8bg4ssoht7ifmvq141nf9vviikir3835mbhvcf2lrmn2r838/family1/6ef59db3da3647d980a4cee3ea13441b 2023-04-09 07:55:35,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__tuv15igsvnpb43q8bg4ssoht7ifmvq141nf9vviikir3835mbhvcf2lrmn2r838/family1/6ef59db3da3647d980a4cee3ea13441b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/0fef8b952db14153a75e863f4c388559_SeqId_28_ 2023-04-09 07:55:35,261 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__tuv15igsvnpb43q8bg4ssoht7ifmvq141nf9vviikir3835mbhvcf2lrmn2r838/family1/6ef59db3da3647d980a4cee3ea13441b into c669f14846d655d587709ca8bd4b7e00/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/0fef8b952db14153a75e863f4c388559_SeqId_28_ - updating store file list. 2023-04-09 07:55:35,264 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/0fef8b952db14153a75e863f4c388559_SeqId_28_ into c669f14846d655d587709ca8bd4b7e00/family1 2023-04-09 07:55:35,264 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__tuv15igsvnpb43q8bg4ssoht7ifmvq141nf9vviikir3835mbhvcf2lrmn2r838/family1/6ef59db3da3647d980a4cee3ea13441b into c669f14846d655d587709ca8bd4b7e00/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/family1/0fef8b952db14153a75e863f4c388559_SeqId_28_) 2023-04-09 07:55:35,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__tuv15igsvnpb43q8bg4ssoht7ifmvq141nf9vviikir3835mbhvcf2lrmn2r838/family1/6ef59db3da3647d980a4cee3ea13441b 2023-04-09 07:55:35,267 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:35,267 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:35,272 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409fc8e156c43f448aba20bfa1e1bfba498 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230409fc8e156c43f448aba20bfa1e1bfba498 2023-04-09 07:55:35,272 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-09 07:55:35,272 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230409' 2023-04-09 07:55:35,273 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:35,291 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/family1/bd3dee16c486425793f027f5a6764bf7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-04-09 07:55:35,292 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/family1/bd3dee16c486425793f027f5a6764bf7}] 2023-04-09 07:55:35,294 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/family1/bd3dee16c486425793f027f5a6764bf7 for inclusion in 410e3eed8a4051a7a5cebb43c08ccfed/family1 2023-04-09 07:55:35,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-04-09 07:55:35,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last=1 2023-04-09 07:55:35,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:55:35,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230409/family1/bd3dee16c486425793f027f5a6764bf7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__j5ub3i1i01e5sojgk1nvhfk1baqd50rhns4o6qpnd9dtml1b3m0srn54u1uv2mb1/family1/bd3dee16c486425793f027f5a6764bf7 2023-04-09 07:55:35,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__j5ub3i1i01e5sojgk1nvhfk1baqd50rhns4o6qpnd9dtml1b3m0srn54u1uv2mb1/family1/bd3dee16c486425793f027f5a6764bf7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0dc7a466046e40ff96e9793e6f2e640e_SeqId_28_ 2023-04-09 07:55:35,299 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__j5ub3i1i01e5sojgk1nvhfk1baqd50rhns4o6qpnd9dtml1b3m0srn54u1uv2mb1/family1/bd3dee16c486425793f027f5a6764bf7 into 410e3eed8a4051a7a5cebb43c08ccfed/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0dc7a466046e40ff96e9793e6f2e640e_SeqId_28_ - updating store file list. 2023-04-09 07:55:35,301 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0dc7a466046e40ff96e9793e6f2e640e_SeqId_28_ into 410e3eed8a4051a7a5cebb43c08ccfed/family1 2023-04-09 07:55:35,301 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__j5ub3i1i01e5sojgk1nvhfk1baqd50rhns4o6qpnd9dtml1b3m0srn54u1uv2mb1/family1/bd3dee16c486425793f027f5a6764bf7 into 410e3eed8a4051a7a5cebb43c08ccfed/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/family1/0dc7a466046e40ff96e9793e6f2e640e_SeqId_28_) 2023-04-09 07:55:35,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__ns_testMinorCompaction__j5ub3i1i01e5sojgk1nvhfk1baqd50rhns4o6qpnd9dtml1b3m0srn54u1uv2mb1/family1/bd3dee16c486425793f027f5a6764bf7 2023-04-09 07:55:35,303 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:35,304 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:35,305 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023040974a4402cee9441b99e86f6ec7fb9299b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023040974a4402cee9441b99e86f6ec7fb9299b 2023-04-09 07:55:35,305 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-04-09 07:55:35,305 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230409' 2023-04-09 07:55:35,305 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230409 for table='ns:testMinorCompaction' and column='family1' 2023-04-09 07:55:35,305 INFO [Listener at localhost/35977] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-04-09 07:55:35,308 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:35,309 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:35,312 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:35,313 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:35,332 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=255 (was 287), OpenFileDescriptor=652 (was 759), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=349 (was 286) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1501 (was 1478) - AvailableMemoryMB LEAK? - 2023-04-09 07:55:35,347 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=349, ProcessCount=169, AvailableMemoryMB=1501 2023-04-09 07:55:35,349 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-04-09 07:55:35,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:35,351 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:55:35,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-04-09 07:55:35,352 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:55:35,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-09 07:55:35,353 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb 2023-04-09 07:55:35,353 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb empty. 2023-04-09 07:55:35,354 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb 2023-04-09 07:55:35,354 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-04-09 07:55:35,361 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-04-09 07:55:35,362 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 74a18d993537627879458039595a67bb, NAME => 'testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:35,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-09 07:55:35,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 74a18d993537627879458039595a67bb, disabling compactions & flushes 2023-04-09 07:55:35,768 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. after waiting 0 ms 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:35,768 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:35,768 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:35,770 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:55:35,771 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026935771"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026935771"}]},"ts":"1681026935771"} 2023-04-09 07:55:35,773 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-09 07:55:35,773 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:55:35,774 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026935774"}]},"ts":"1681026935774"} 2023-04-09 07:55:35,775 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-04-09 07:55:35,778 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=74a18d993537627879458039595a67bb, ASSIGN}] 2023-04-09 07:55:35,780 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=74a18d993537627879458039595a67bb, ASSIGN 2023-04-09 07:55:35,781 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=74a18d993537627879458039595a67bb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:35,932 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=74a18d993537627879458039595a67bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:35,932 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026935932"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026935932"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026935932"}]},"ts":"1681026935932"} 2023-04-09 07:55:35,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 74a18d993537627879458039595a67bb, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:35,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-09 07:55:36,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:36,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 74a18d993537627879458039595a67bb, NAME => 'testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.', STARTKEY => '', ENDKEY => ''} 2023-04-09 07:55:36,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 74a18d993537627879458039595a67bb 2023-04-09 07:55:36,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:36,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 74a18d993537627879458039595a67bb 2023-04-09 07:55:36,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 74a18d993537627879458039595a67bb 2023-04-09 07:55:36,100 INFO [StoreOpener-74a18d993537627879458039595a67bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 74a18d993537627879458039595a67bb 2023-04-09 07:55:36,102 DEBUG [StoreOpener-74a18d993537627879458039595a67bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:36,102 DEBUG [StoreOpener-74a18d993537627879458039595a67bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:36,102 INFO [StoreOpener-74a18d993537627879458039595a67bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74a18d993537627879458039595a67bb columnFamilyName family1 2023-04-09 07:55:36,109 INFO [StoreOpener-74a18d993537627879458039595a67bb-1] regionserver.HStore(310): Store=74a18d993537627879458039595a67bb/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:36,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb 2023-04-09 07:55:36,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb 2023-04-09 07:55:36,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 74a18d993537627879458039595a67bb 2023-04-09 07:55:36,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:36,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 74a18d993537627879458039595a67bb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70933863, jitterRate=0.05699692666530609}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-09 07:55:36,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:36,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., pid=114, masterSystemTime=1681026936086 2023-04-09 07:55:36,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:36,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:36,126 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=74a18d993537627879458039595a67bb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,126 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1681026936126"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026936126"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026936126"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026936126"}]},"ts":"1681026936126"} 2023-04-09 07:55:36,129 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-04-09 07:55:36,129 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 74a18d993537627879458039595a67bb, server=jenkins-hbase4.apache.org,34907,1681026816245 in 193 msec 2023-04-09 07:55:36,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-04-09 07:55:36,131 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=74a18d993537627879458039595a67bb, ASSIGN in 351 msec 2023-04-09 07:55:36,131 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:55:36,132 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026936131"}]},"ts":"1681026936131"} 2023-04-09 07:55:36,133 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-04-09 07:55:36,136 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:55:36,137 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 787 msec 2023-04-09 07:55:36,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-04-09 07:55:36,459 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-04-09 07:55:36,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:36,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,487 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:36,487 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:36,487 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:36,487 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:36,488 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,488 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,490 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,490 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:36,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:36,491 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,491 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,492 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:36,492 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:36,492 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:36,496 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:36,496 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:36,496 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:36,496 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:36,497 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-04-09 07:55:36,530 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113065162b5e56d548ffb73bb2ded9f2cd27 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113065162b5e56d548ffb73bb2ded9f2cd27 2023-04-09 07:55:36,934 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2db3d53ff9324c7da3b7bd7182b549dc 2023-04-09 07:55:36,940 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2db3d53ff9324c7da3b7bd7182b549dc as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc 2023-04-09 07:55:36,944 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc, entries=1000, sequenceid=5, filesize=166.0 K 2023-04-09 07:55:36,945 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 448ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:36,945 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:36,946 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:36,946 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:36,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:36,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:36,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:36,946 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:36,948 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:36,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:36,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,948 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:36,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:36,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:36,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:36,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:36,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bd5a66d[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:36,950 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:36,950 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,952 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:36,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:36,952 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:36,952 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,952 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:36,954 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:36,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:36,954 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:36,954 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:36,954 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:36,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:36,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:36,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:36,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:36,959 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:36,959 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:36,959 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:36,959 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,959 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:36,962 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,962 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:36,962 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:36,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:36,962 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:36,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:36,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:36,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:36,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:36,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:36,967 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:36,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:36,967 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:36,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:36,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:36,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:36,967 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:37,068 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:37,068 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:37,072 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:37,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:37,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,074 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:37,074 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:37,074 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:37,074 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:37,074 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,074 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,075 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:37,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:37,076 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,076 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:37,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,077 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,077 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:37,077 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:37,077 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:37,077 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:37,077 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=74 B heapSize=408 B 2023-04-09 07:55:37,088 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112840a6a09e532e4e8c8bc113dbd932cb1c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112840a6a09e532e4e8c8bc113dbd932cb1c 2023-04-09 07:55:37,095 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/11986eb9af964387ba9ccaf24e9c7987 2023-04-09 07:55:37,100 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/11986eb9af964387ba9ccaf24e9c7987 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987 2023-04-09 07:55:37,104 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987, entries=1, sequenceid=9, filesize=5.0 K 2023-04-09 07:55:37,105 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 28ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:37,105 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:37,105 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:37,105 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:37,105 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:37,105 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:37,106 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:37,107 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,107 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:37,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:37,108 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,108 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:37,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:37,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7adc6745[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:37,109 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:37,109 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,111 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,111 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:37,111 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:37,111 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:37,111 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,112 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:37,113 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:37,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:37,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:37,113 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:37,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:37,116 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:37,116 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:37,116 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:37,116 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,116 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:37,118 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:37,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:37,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:37,118 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,120 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:37,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,122 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:37,122 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:37,123 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:37,123 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,123 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,123 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:37,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:37,123 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:37,123 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:37,223 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:37,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:37,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:37,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:37,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,231 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:37,231 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:37,231 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:37,231 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:37,232 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,232 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,236 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:37,236 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:37,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:37,237 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:37,237 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:37,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:37,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:37,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:37,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:37,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:37,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:37,238 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:37,238 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:37,238 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:37,238 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:37,451 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-09 07:55:37,656 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512016f35377873dc43a59d8a7e956c0aa445 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016f35377873dc43a59d8a7e956c0aa445 2023-04-09 07:55:38,062 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2abe434e928d473d95bfb67518b9fb1e 2023-04-09 07:55:38,066 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2abe434e928d473d95bfb67518b9fb1e as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e 2023-04-09 07:55:38,070 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e, entries=1, sequenceid=13, filesize=4.9 K 2023-04-09 07:55:38,070 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 832ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:38,071 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:38,071 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:38,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:38,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:38,071 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,072 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:38,072 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:38,073 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:38,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,073 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:38,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:38,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a2dd93[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:38,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:38,075 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,076 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,076 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:38,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:38,077 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:38,077 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,077 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:38,078 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:38,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:38,078 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:38,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:38,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:38,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:38,080 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:38,080 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:38,080 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:38,080 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,080 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:38,083 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,083 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:38,083 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:38,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,084 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:38,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:38,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,086 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,089 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,089 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,089 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,089 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:38,089 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,090 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:38,090 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:38,090 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:38,090 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:38,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:38,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:38,190 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:38,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:38,196 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:38,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:38,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,200 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:38,200 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:38,200 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:38,200 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:38,201 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,201 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,202 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,202 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:38,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:38,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:38,202 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:38,202 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:38,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:38,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,204 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:38,204 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:38,204 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:38,204 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:38,204 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:38,204 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:38,621 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120579eca4ee55c14c15b87b78b2649d6829 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120579eca4ee55c14c15b87b78b2649d6829 2023-04-09 07:55:39,026 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/71960960433045799802c9237f6e8335 2023-04-09 07:55:39,030 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/71960960433045799802c9237f6e8335 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335 2023-04-09 07:55:39,033 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335, entries=1, sequenceid=17, filesize=4.9 K 2023-04-09 07:55:39,034 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 830ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:39,034 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:39,034 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:39,034 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:39,034 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:39,034 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,035 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:39,035 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:39,036 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,036 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:39,037 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,037 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:39,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:39,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72224f4b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:39,038 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:39,038 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,040 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,041 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:39,041 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:39,041 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:39,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:39,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:39,043 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,043 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:39,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:39,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:39,044 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:39,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:39,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:39,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:39,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:39,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:39,047 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:39,047 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:39,047 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:39,047 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,047 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:39,050 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,050 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,050 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:39,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:39,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:39,050 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:39,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,052 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,054 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,054 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:39,054 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:39,054 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:39,054 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:39,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:39,054 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:39,055 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:39,054 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:39,055 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:39,155 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:39,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:39,160 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:39,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:39,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,162 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:39,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:39,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:39,162 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:39,163 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,163 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,165 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:39,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:39,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:39,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:39,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,166 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,167 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:39,167 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:39,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:39,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:39,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:39,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:39,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:39,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:39,169 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:39,169 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:39,169 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:39,170 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:39,590 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228f6cf47665b8a4be997a67e3e6d0ea1a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6cf47665b8a4be997a67e3e6d0ea1a7 2023-04-09 07:55:39,995 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/991d4b62e71c419eb4581b6098e77cbb 2023-04-09 07:55:39,999 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/991d4b62e71c419eb4581b6098e77cbb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb 2023-04-09 07:55:40,003 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb, entries=1, sequenceid=21, filesize=4.9 K 2023-04-09 07:55:40,004 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 834ms, sequenceid=21, compaction requested=true 2023-04-09 07:55:40,004 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:40,004 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:40,006 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,006 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:40,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:40,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@447269ac[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:40,008 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:40,008 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,009 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,009 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:40,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:40,010 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:40,010 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,010 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:40,011 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,011 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,011 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,011 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:40,013 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:40,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:40,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:40,013 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,013 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:40,016 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,016 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,016 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,017 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,018 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,023 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:40,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,023 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:40,024 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:40,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,124 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:40,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:40,129 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:40,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:40,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,130 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,130 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,130 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:40,130 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:40,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,131 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,133 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,133 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:40,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,134 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:40,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,135 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:40,135 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,135 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:40,135 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,135 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:40,135 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:40,135 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:40,135 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,135 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:40,135 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:40,553 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123139313f4eecb649fd8d7588f559a512e3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123139313f4eecb649fd8d7588f559a512e3 2023-04-09 07:55:40,558 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/aedb9a9b40954afcadb3508a3accbe06 2023-04-09 07:55:40,561 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/aedb9a9b40954afcadb3508a3accbe06 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06 2023-04-09 07:55:40,564 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06, entries=1, sequenceid=25, filesize=4.9 K 2023-04-09 07:55:40,565 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 430ms, sequenceid=25, compaction requested=true 2023-04-09 07:55:40,565 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:40,565 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,566 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:40,566 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:40,568 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,568 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,569 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,569 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:40,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d0dd828[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:40,570 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:40,571 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,572 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,572 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:40,572 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:40,572 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:40,572 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,572 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:40,573 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:40,573 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,573 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,573 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:40,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:40,575 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:40,576 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:40,576 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:40,576 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,576 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:40,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,577 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,578 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,579 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:40,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,582 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:40,582 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,582 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,583 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,583 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,583 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,583 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,683 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:40,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:40,687 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:40,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:40,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,689 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:40,689 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:40,690 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,690 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,691 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,691 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,691 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,691 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:40,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,692 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:40,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,692 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,692 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:40,692 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,692 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:40,692 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,692 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:40,693 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:40,693 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:40,693 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,693 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:40,693 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:40,704 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601018834ad41a65d48b398ac6f92f978a321 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601018834ad41a65d48b398ac6f92f978a321 2023-04-09 07:55:40,709 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/f6380122520d4278a9e13a88820d7d9d 2023-04-09 07:55:40,712 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/f6380122520d4278a9e13a88820d7d9d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d 2023-04-09 07:55:40,716 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d, entries=1, sequenceid=29, filesize=4.9 K 2023-04-09 07:55:40,716 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 23ms, sequenceid=29, compaction requested=true 2023-04-09 07:55:40,716 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:40,716 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,716 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:40,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:40,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:40,717 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:40,718 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,718 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,719 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,719 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:40,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f51d85f[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:40,720 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:40,720 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,721 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:40,721 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:40,721 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,721 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:40,724 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,724 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:40,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:40,726 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:40,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:40,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:40,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,726 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:40,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,730 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,732 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:40,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,732 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:40,733 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:40,833 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:40,833 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:40,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:40,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:40,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,840 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,840 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,841 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:40,841 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:40,841 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,841 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,842 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,842 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,842 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,843 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,843 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:40,843 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:40,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:40,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:40,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:40,844 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:40,844 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,844 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:40,844 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:40,863 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a9c3e3df936541e883519ae2891a5e2f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9c3e3df936541e883519ae2891a5e2f 2023-04-09 07:55:40,881 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2bb43b42043b43678fc90ca93d67c9fd 2023-04-09 07:55:40,885 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2bb43b42043b43678fc90ca93d67c9fd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd 2023-04-09 07:55:40,888 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd, entries=1, sequenceid=33, filesize=4.9 K 2023-04-09 07:55:40,889 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 45ms, sequenceid=33, compaction requested=true 2023-04-09 07:55:40,889 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:40,889 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:40,889 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:40,889 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:40,890 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,890 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:40,890 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:40,892 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,892 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,893 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,893 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:40,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:40,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f699ee9[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:40,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:40,897 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,898 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:40,898 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:40,899 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,899 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,900 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:40,900 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:40,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:40,903 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:40,903 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:40,903 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:40,903 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,903 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:40,904 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,905 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:40,906 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:40,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:40,908 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:40,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:40,925 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:40,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:40,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:40,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:40,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,928 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:40,928 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:40,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:40,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:40,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:40,929 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:40,929 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:41,029 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:41,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:41,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:41,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:41,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,041 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:41,041 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:41,041 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:41,041 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:41,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,042 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,043 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,043 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:41,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:41,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:41,043 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,043 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:41,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:41,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,045 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:41,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,045 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:41,045 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:41,045 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:41,052 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:41,052 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:41,052 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:41,052 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:41,053 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:41,083 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079d920fb9c91a444aadfb95872072f951 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509079d920fb9c91a444aadfb95872072f951 2023-04-09 07:55:41,088 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/43bae7ce31114a05a10f7e8128691edc 2023-04-09 07:55:41,102 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/43bae7ce31114a05a10f7e8128691edc as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc 2023-04-09 07:55:41,109 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc, entries=1, sequenceid=37, filesize=4.9 K 2023-04-09 07:55:41,110 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 57ms, sequenceid=37, compaction requested=true 2023-04-09 07:55:41,110 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:41,110 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:41,110 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:41,110 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:41,110 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,110 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:41,110 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:41,111 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,112 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:41,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,112 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:41,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:41,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:41,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:41,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:41,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7801ada1[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:41,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:41,114 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,129 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:41,130 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:41,130 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,130 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:41,130 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32f4cb3b to 127.0.0.1:57867 2023-04-09 07:55:41,130 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:41,131 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:41,131 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3894c415 to 127.0.0.1:57867 2023-04-09 07:55:41,131 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:41,132 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:41,132 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:41,132 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:41,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:41,132 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:41,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:41,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:41,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:41,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:41,236 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:41,236 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:41,236 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:41,236 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,236 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:41,238 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,238 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:41,238 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:41,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:41,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:41,239 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:41,240 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:41,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:41,242 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:41,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:41,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:41,242 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:41,243 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:41,242 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,243 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,243 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,243 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:41,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:41,243 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:41,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:41,243 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:41,344 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:41,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:41,348 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:41,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:41,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,350 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:41,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:41,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-04-09 07:55:41,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-04-09 07:55:41,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,350 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,351 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,351 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:41,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:41,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:41,351 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:41,351 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,352 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-04-09 07:55:41,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:41,352 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-04-09 07:55:41,352 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:41,353 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-04-09 07:55:41,353 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-09 07:55:41,353 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:41,353 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:41,353 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. started... 2023-04-09 07:55:41,353 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 74a18d993537627879458039595a67bb 1/1 column families, dataSize=73 B heapSize=408 B 2023-04-09 07:55:41,689 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:41,762 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200a227bf315f64bb7b98fc4dc23a7a385 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200a227bf315f64bb7b98fc4dc23a7a385 2023-04-09 07:55:42,167 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/83a258ccd24044e89fdf78484e0f09dd 2023-04-09 07:55:42,171 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/83a258ccd24044e89fdf78484e0f09dd as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd 2023-04-09 07:55:42,173 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd, entries=1, sequenceid=41, filesize=4.9 K 2023-04-09 07:55:42,174 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 74a18d993537627879458039595a67bb in 821ms, sequenceid=41, compaction requested=true 2023-04-09 07:55:42,174 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:42,174 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:42,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-09 07:55:42,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-09 07:55:42,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-04-09 07:55:42,174 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-04-09 07:55:42,176 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:42,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,176 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:42,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:42,177 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-04-09 07:55:42,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bfef611[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:42,178 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-04-09 07:55:42,178 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,178 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,179 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-04-09 07:55:42,179 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-04-09 07:55:42,179 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,179 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:42,179 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:42,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-04-09 07:55:42,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:42,180 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,180 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-04-09 07:55:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:42,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:42,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:42,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:42,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-04-09 07:55:42,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-04-09 07:55:42,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:42,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:42,182 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,182 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:42,184 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,184 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:42,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,184 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:42,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:42,184 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,185 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:42,187 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-04-09 07:55:42,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-04-09 07:55:42,187 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:42,187 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:42,187 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:42,188 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:42,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:42,188 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,188 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,188 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:42,188 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:42,288 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:42,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:42,290 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-04-09 07:55:42,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:55:42,292 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:55:42,292 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:55:42,292 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027542292 2023-04-09 07:55:42,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-04-09 07:55:42,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-04-09 07:55:42,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:42,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x0471502c to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:42,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e288293, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:42,298 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:43,124 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037545300e271f49b29826f196debc85d0 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601037545300e271f49b29826f196debc85d0 2023-04-09 07:55:43,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:43,137 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:43,139 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38766, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:43,144 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:55:43,145 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:55:43,151 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 first=Optional[r5] last=Optional[r6] 2023-04-09 07:55:43,152 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cd3f36d33a2d4c2c91c1998ef70d2e31}] 2023-04-09 07:55:43,158 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 for inclusion in 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:43,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-04-09 07:55:43,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:55:43,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:43,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lpd7ksd4lbtsjccjhdtsj8r4620s9ca3o59n6d6ka4he0mgu9a6n9413orl4vdmu/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 2023-04-09 07:55:43,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lpd7ksd4lbtsjccjhdtsj8r4620s9ca3o59n6d6ka4he0mgu9a6n9413orl4vdmu/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_ 2023-04-09 07:55:43,162 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lpd7ksd4lbtsjccjhdtsj8r4620s9ca3o59n6d6ka4he0mgu9a6n9413orl4vdmu/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 into 74a18d993537627879458039595a67bb/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_ - updating store file list. 2023-04-09 07:55:43,165 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_ into 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:43,165 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lpd7ksd4lbtsjccjhdtsj8r4620s9ca3o59n6d6ka4he0mgu9a6n9413orl4vdmu/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 into 74a18d993537627879458039595a67bb/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_) 2023-04-09 07:55:43,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__lpd7ksd4lbtsjccjhdtsj8r4620s9ca3o59n6d6ka4he0mgu9a6n9413orl4vdmu/family1/cd3f36d33a2d4c2c91c1998ef70d2e31 2023-04-09 07:55:43,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-04-09 07:55:43,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-04-09 07:55:43,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): 74a18d993537627879458039595a67bb/family1 is initiating minor compaction 2023-04-09 07:55:43,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@276320be; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:43,167 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 74a18d993537627879458039595a67bb/family1 in testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:43,167 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp, totalSize=49.7 K 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 11986eb9af964387ba9ccaf24e9c7987, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 2abe434e928d473d95bfb67518b9fb1e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 71960960433045799802c9237f6e8335, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 991d4b62e71c419eb4581b6098e77cbb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting aedb9a9b40954afcadb3508a3accbe06, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-04-09 07:55:43,167 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting f6380122520d4278a9e13a88820d7d9d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-04-09 07:55:43,168 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 2bb43b42043b43678fc90ca93d67c9fd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-04-09 07:55:43,168 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 43bae7ce31114a05a10f7e8128691edc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-04-09 07:55:43,168 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 83a258ccd24044e89fdf78484e0f09dd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-04-09 07:55:43,168 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting c45e450f98274491993acf5afc962ffd_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-04-09 07:55:43,169 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:43,169 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:43,170 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601018834ad41a65d48b398ac6f92f978a321 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601018834ad41a65d48b398ac6f92f978a321 2023-04-09 07:55:43,171 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9c3e3df936541e883519ae2891a5e2f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103a9c3e3df936541e883519ae2891a5e2f 2023-04-09 07:55:43,171 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-04-09 07:55:43,172 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-04-09 07:55:43,172 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:43,172 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:43,187 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74a18d993537627879458039595a67bb#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:43,998 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/3979d2f1af16465b89755e87737901fb as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3979d2f1af16465b89755e87737901fb 2023-04-09 07:55:43,999 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512318839a77f89de4aa48f804326c3282a10 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512318839a77f89de4aa48f804326c3282a10 2023-04-09 07:55:43,999 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:44,002 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 74a18d993537627879458039595a67bb/family1 of 74a18d993537627879458039595a67bb into 3979d2f1af16465b89755e87737901fb(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:44,002 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:44,002 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1681026943167; duration=0sec 2023-04-09 07:55:44,003 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:44,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_] to archive 2023-04-09 07:55:44,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:44,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/11986eb9af964387ba9ccaf24e9c7987 2023-04-09 07:55:44,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2abe434e928d473d95bfb67518b9fb1e 2023-04-09 07:55:44,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/71960960433045799802c9237f6e8335 2023-04-09 07:55:44,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/991d4b62e71c419eb4581b6098e77cbb 2023-04-09 07:55:44,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/aedb9a9b40954afcadb3508a3accbe06 2023-04-09 07:55:44,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f6380122520d4278a9e13a88820d7d9d 2023-04-09 07:55:44,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bb43b42043b43678fc90ca93d67c9fd 2023-04-09 07:55:44,015 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/43bae7ce31114a05a10f7e8128691edc 2023-04-09 07:55:44,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/83a258ccd24044e89fdf78484e0f09dd 2023-04-09 07:55:44,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/c45e450f98274491993acf5afc962ffd_SeqId_44_ 2023-04-09 07:55:44,023 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 first=Optional[r1] last=Optional[r4] 2023-04-09 07:55:44,024 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ceb1904d7b9f4b6db251684cdb5bc0d6}] 2023-04-09 07:55:44,026 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 for inclusion in 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:44,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-04-09 07:55:44,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:55:44,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:44,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__tjhl3cd97ev3d70hviq3v3ls1vv42sidj0ihju5iu02qq2qc2tds96j73ice73k6/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 2023-04-09 07:55:44,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__tjhl3cd97ev3d70hviq3v3ls1vv42sidj0ihju5iu02qq2qc2tds96j73ice73k6/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_ 2023-04-09 07:55:44,030 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__tjhl3cd97ev3d70hviq3v3ls1vv42sidj0ihju5iu02qq2qc2tds96j73ice73k6/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 into 74a18d993537627879458039595a67bb/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_ - updating store file list. 2023-04-09 07:55:44,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_ into 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:44,034 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__tjhl3cd97ev3d70hviq3v3ls1vv42sidj0ihju5iu02qq2qc2tds96j73ice73k6/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 into 74a18d993537627879458039595a67bb/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_) 2023-04-09 07:55:44,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__tjhl3cd97ev3d70hviq3v3ls1vv42sidj0ihju5iu02qq2qc2tds96j73ice73k6/family1/ceb1904d7b9f4b6db251684cdb5bc0d6 2023-04-09 07:55:44,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-04-09 07:55:44,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-04-09 07:55:44,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(1912): 74a18d993537627879458039595a67bb/family1 is initiating minor compaction (all files) 2023-04-09 07:55:44,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4fa7c4ba; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:44,036 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 74a18d993537627879458039595a67bb/family1 in testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:55:44,036 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3979d2f1af16465b89755e87737901fb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp, totalSize=178.1 K 2023-04-09 07:55:44,036 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 2db3d53ff9324c7da3b7bd7182b549dc, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840049 2023-04-09 07:55:44,037 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting 3979d2f1af16465b89755e87737901fb, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240049 2023-04-09 07:55:44,037 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] compactions.Compactor(207): Compacting f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240049 2023-04-09 07:55:44,038 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:44,039 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:44,040 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016f35377873dc43a59d8a7e956c0aa445 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512016f35377873dc43a59d8a7e956c0aa445 2023-04-09 07:55:44,042 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120579eca4ee55c14c15b87b78b2649d6829 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120579eca4ee55c14c15b87b78b2649d6829 2023-04-09 07:55:44,043 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6cf47665b8a4be997a67e3e6d0ea1a7 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151228f6cf47665b8a4be997a67e3e6d0ea1a7 2023-04-09 07:55:44,044 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123139313f4eecb649fd8d7588f559a512e3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123139313f4eecb649fd8d7588f559a512e3 2023-04-09 07:55:44,044 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-04-09 07:55:44,044 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-04-09 07:55:44,044 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:44,045 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:44,053 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74a18d993537627879458039595a67bb#family1#compaction#166 average throughput is 17.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:44,879 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a3680a7ff731434d9524a53c01682d4f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a3680a7ff731434d9524a53c01682d4f 2023-04-09 07:55:44,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:44,899 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/273f50f0dc2648eb82d86c02ffec2aff first=Optional[r00] last=Optional[r8] 2023-04-09 07:55:44,900 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/273f50f0dc2648eb82d86c02ffec2aff}] 2023-04-09 07:55:44,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/273f50f0dc2648eb82d86c02ffec2aff for inclusion in 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:44,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-04-09 07:55:44,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last= 2023-04-09 07:55:44,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:44,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/273f50f0dc2648eb82d86c02ffec2aff to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__v70gro8obh23o34jm579i7t7jon8t2i2rbq65fglf7ad7qb5gm6p6nv376h6v139/family1/273f50f0dc2648eb82d86c02ffec2aff 2023-04-09 07:55:44,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__v70gro8obh23o34jm579i7t7jon8t2i2rbq65fglf7ad7qb5gm6p6nv376h6v139/family1/273f50f0dc2648eb82d86c02ffec2aff as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3b65793cd6254b7a82e64bd7eb5e9c35_SeqId_49_ 2023-04-09 07:55:44,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__v70gro8obh23o34jm579i7t7jon8t2i2rbq65fglf7ad7qb5gm6p6nv376h6v139/family1/273f50f0dc2648eb82d86c02ffec2aff into 74a18d993537627879458039595a67bb/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3b65793cd6254b7a82e64bd7eb5e9c35_SeqId_49_ - updating store file list. 2023-04-09 07:55:44,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3b65793cd6254b7a82e64bd7eb5e9c35_SeqId_49_ into 74a18d993537627879458039595a67bb/family1 2023-04-09 07:55:44,909 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__v70gro8obh23o34jm579i7t7jon8t2i2rbq65fglf7ad7qb5gm6p6nv376h6v139/family1/273f50f0dc2648eb82d86c02ffec2aff into 74a18d993537627879458039595a67bb/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3b65793cd6254b7a82e64bd7eb5e9c35_SeqId_49_) 2023-04-09 07:55:44,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMinorCompactionWithMonthlyPolicy__v70gro8obh23o34jm579i7t7jon8t2i2rbq65fglf7ad7qb5gm6p6nv376h6v139/family1/273f50f0dc2648eb82d86c02ffec2aff 2023-04-09 07:55:44,910 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:44,911 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:44,912 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200a227bf315f64bb7b98fc4dc23a7a385 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200a227bf315f64bb7b98fc4dc23a7a385 2023-04-09 07:55:44,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112840a6a09e532e4e8c8bc113dbd932cb1c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112840a6a09e532e4e8c8bc113dbd932cb1c 2023-04-09 07:55:44,913 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113065162b5e56d548ffb73bb2ded9f2cd27 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113065162b5e56d548ffb73bb2ded9f2cd27 2023-04-09 07:55:44,913 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-04-09 07:55:44,913 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-04-09 07:55:44,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:44,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:44,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-04-09 07:55:44,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-04-09 07:55:44,914 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:55:44,916 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-04-09 07:55:44,917 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.6260 sec 2023-04-09 07:55:45,104 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=260 (was 255) - Thread LEAK? -, OpenFileDescriptor=655 (was 652) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=325 (was 349), ProcessCount=169 (was 169), AvailableMemoryMB=1518 (was 1501) - AvailableMemoryMB LEAK? - 2023-04-09 07:55:45,118 INFO [Listener at localhost/35977] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=260, OpenFileDescriptor=655, MaxFileDescriptor=60000, SystemLoadAverage=325, ProcessCount=169, AvailableMemoryMB=1516 2023-04-09 07:55:45,138 DEBUG [Listener at localhost/35977] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-04-09 07:55:45,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10&5\x8ABC\xA4\x1EI\xF9\x18"\x07\xFE\xE6\x95\xF7"\x10'g\x98i{\xF6\xD5\xEF\x0Bg\xC4\xF2\x91P\x14\xB6*\x10X%\x16\x05\x90;4\xBD0d!\xBD\x92a\xCE\xF12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-04-09 07:55:45,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-04-09 07:55:45,193 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-04-09 07:55:45,193 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-04-09 07:55:45,194 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-09 07:55:45,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901 empty. 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec empty. 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd empty. 2023-04-09 07:55:45,196 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:45,197 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:45,197 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:45,197 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-04-09 07:55:45,271 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/.tmp/family1/2bbcaca9ab1f4be8af058bf513610c66 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2bbcaca9ab1f4be8af058bf513610c66 2023-04-09 07:55:45,275 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 74a18d993537627879458039595a67bb/family1 of 74a18d993537627879458039595a67bb into 2bbcaca9ab1f4be8af058bf513610c66(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-04-09 07:55:45,275 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:55:45,275 INFO [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1681026944035; duration=1sec 2023-04-09 07:55:45,275 DEBUG [RS:0;jenkins-hbase4:34907-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-09 07:55:45,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:45,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3979d2f1af16465b89755e87737901fb, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_] to archive 2023-04-09 07:55:45,305 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:45,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/2db3d53ff9324c7da3b7bd7182b549dc 2023-04-09 07:55:45,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3979d2f1af16465b89755e87737901fb to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/3979d2f1af16465b89755e87737901fb 2023-04-09 07:55:45,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_ to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/family1/f8e67fe6cbcd4f4085b97cba2f1dc56b_SeqId_47_ 2023-04-09 07:55:45,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:45,603 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-04-09 07:55:45,605 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => ce3a8bff0d6396ea7236f773f152c0ec, NAME => 'testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10&5\x8ABC\xA4\x1EI\xF9\x18"\x07\xFE\xE6\x95\xF7"\x10'g\x98i{\xF6\xD5\xEF\x0Bg\xC4\xF2\x91P\x14\xB6*\x10X%\x16\x05\x90;4\xBD0d!\xBD\x92a\xCE\xF12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:45,605 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => fde6fda2ba4dd6622e7e423c1f8b61dd, NAME => 'testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10&5\x8ABC\xA4\x1EI\xF9\x18"\x07\xFE\xE6\x95\xF7"\x10'g\x98i{\xF6\xD5\xEF\x0Bg\xC4\xF2\x91P\x14\xB6*\x10X%\x16\x05\x90;4\xBD0d!\xBD\x92a\xCE\xF12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:45,605 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => a09e145f7e57c7d9f5ff2cc3dd5d6901, NAME => 'testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10&5\x8ABC\xA4\x1EI\xF9\x18"\x07\xFE\xE6\x95\xF7"\x10'g\x98i{\xF6\xD5\xEF\x0Bg\xC4\xF2\x91P\x14\xB6*\x10X%\x16\x05\x90;4\xBD0d!\xBD\x92a\xCE\xF12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/.tmp 2023-04-09 07:55:45,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing fde6fda2ba4dd6622e7e423c1f8b61dd, disabling compactions & flushes 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing a09e145f7e57c7d9f5ff2cc3dd5d6901, disabling compactions & flushes 2023-04-09 07:55:46,011 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,011 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing ce3a8bff0d6396ea7236f773f152c0ec, disabling compactions & flushes 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. after waiting 0 ms 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,011 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,012 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,012 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. after waiting 0 ms 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,012 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. after waiting 0 ms 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:46,012 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,012 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:46,014 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-04-09 07:55:46,015 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946015"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026946015"}]},"ts":"1681026946015"} 2023-04-09 07:55:46,015 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1681026946015"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026946015"}]},"ts":"1681026946015"} 2023-04-09 07:55:46,015 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946015"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1681026946015"}]},"ts":"1681026946015"} 2023-04-09 07:55:46,017 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-04-09 07:55:46,017 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-09 07:55:46,017 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026946017"}]},"ts":"1681026946017"} 2023-04-09 07:55:46,018 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-04-09 07:55:46,021 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a09e145f7e57c7d9f5ff2cc3dd5d6901, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce3a8bff0d6396ea7236f773f152c0ec, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fde6fda2ba4dd6622e7e423c1f8b61dd, ASSIGN}] 2023-04-09 07:55:46,022 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fde6fda2ba4dd6622e7e423c1f8b61dd, ASSIGN 2023-04-09 07:55:46,022 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce3a8bff0d6396ea7236f773f152c0ec, ASSIGN 2023-04-09 07:55:46,022 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a09e145f7e57c7d9f5ff2cc3dd5d6901, ASSIGN 2023-04-09 07:55:46,023 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fde6fda2ba4dd6622e7e423c1f8b61dd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:46,023 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a09e145f7e57c7d9f5ff2cc3dd5d6901, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:46,023 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce3a8bff0d6396ea7236f773f152c0ec, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34907,1681026816245; forceNewPlan=false, retain=false 2023-04-09 07:55:46,174 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=a09e145f7e57c7d9f5ff2cc3dd5d6901, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,174 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=fde6fda2ba4dd6622e7e423c1f8b61dd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,174 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ce3a8bff0d6396ea7236f773f152c0ec, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,175 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946174"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026946174"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026946174"}]},"ts":"1681026946174"} 2023-04-09 07:55:46,175 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946174"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026946174"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026946174"}]},"ts":"1681026946174"} 2023-04-09 07:55:46,175 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1681026946174"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1681026946174"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1681026946174"}]},"ts":"1681026946174"} 2023-04-09 07:55:46,176 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure a09e145f7e57c7d9f5ff2cc3dd5d6901, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:46,177 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=119, state=RUNNABLE; OpenRegionProcedure fde6fda2ba4dd6622e7e423c1f8b61dd, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:46,177 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=118, state=RUNNABLE; OpenRegionProcedure ce3a8bff0d6396ea7236f773f152c0ec, server=jenkins-hbase4.apache.org,34907,1681026816245}] 2023-04-09 07:55:46,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:46,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a09e145f7e57c7d9f5ff2cc3dd5d6901, NAME => 'testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.', STARTKEY => '', ENDKEY => '1'} 2023-04-09 07:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,330 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,351 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,353 DEBUG [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 2023-04-09 07:55:46,353 DEBUG [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 2023-04-09 07:55:46,353 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a09e145f7e57c7d9f5ff2cc3dd5d6901 columnFamilyName family1 2023-04-09 07:55:46,354 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] regionserver.HStore(310): Store=a09e145f7e57c7d9f5ff2cc3dd5d6901/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,354 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,355 DEBUG [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2 2023-04-09 07:55:46,355 DEBUG [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2 2023-04-09 07:55:46,355 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a09e145f7e57c7d9f5ff2cc3dd5d6901 columnFamilyName family2 2023-04-09 07:55:46,356 INFO [StoreOpener-a09e145f7e57c7d9f5ff2cc3dd5d6901-1] regionserver.HStore(310): Store=a09e145f7e57c7d9f5ff2cc3dd5d6901/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:46,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:55:46,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:46,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a09e145f7e57c7d9f5ff2cc3dd5d6901; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74096130, jitterRate=0.10411837697029114}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:46,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:46,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901., pid=120, masterSystemTime=1681026946327 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:46,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ce3a8bff0d6396ea7236f773f152c0ec, NAME => 'testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.', STARTKEY => '1', ENDKEY => '2'} 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,364 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=a09e145f7e57c7d9f5ff2cc3dd5d6901, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,364 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946364"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026946364"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026946364"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026946364"}]},"ts":"1681026946364"} 2023-04-09 07:55:46,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,367 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-04-09 07:55:46,367 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure a09e145f7e57c7d9f5ff2cc3dd5d6901, server=jenkins-hbase4.apache.org,34907,1681026816245 in 189 msec 2023-04-09 07:55:46,368 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a09e145f7e57c7d9f5ff2cc3dd5d6901, ASSIGN in 346 msec 2023-04-09 07:55:46,387 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,388 DEBUG [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1 2023-04-09 07:55:46,388 DEBUG [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1 2023-04-09 07:55:46,389 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ce3a8bff0d6396ea7236f773f152c0ec columnFamilyName family1 2023-04-09 07:55:46,390 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] regionserver.HStore(310): Store=ce3a8bff0d6396ea7236f773f152c0ec/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,390 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,391 DEBUG [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2 2023-04-09 07:55:46,391 DEBUG [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2 2023-04-09 07:55:46,391 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ce3a8bff0d6396ea7236f773f152c0ec columnFamilyName family2 2023-04-09 07:55:46,392 INFO [StoreOpener-ce3a8bff0d6396ea7236f773f152c0ec-1] regionserver.HStore(310): Store=ce3a8bff0d6396ea7236f773f152c0ec/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:46,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:55:46,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:46,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ce3a8bff0d6396ea7236f773f152c0ec; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62859793, jitterRate=-0.06331609189510345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:46,397 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:46,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec., pid=122, masterSystemTime=1681026946327 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:46,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fde6fda2ba4dd6622e7e423c1f8b61dd, NAME => 'testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.', STARTKEY => '2', ENDKEY => ''} 2023-04-09 07:55:46,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ce3a8bff0d6396ea7236f773f152c0ec, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-09 07:55:46,399 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1681026946399"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026946399"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026946399"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026946399"}]},"ts":"1681026946399"} 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=118 2023-04-09 07:55:46,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=118, state=SUCCESS; OpenRegionProcedure ce3a8bff0d6396ea7236f773f152c0ec, server=jenkins-hbase4.apache.org,34907,1681026816245 in 223 msec 2023-04-09 07:55:46,402 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ce3a8bff0d6396ea7236f773f152c0ec, ASSIGN in 380 msec 2023-04-09 07:55:46,425 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,426 DEBUG [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1 2023-04-09 07:55:46,426 DEBUG [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1 2023-04-09 07:55:46,426 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fde6fda2ba4dd6622e7e423c1f8b61dd columnFamilyName family1 2023-04-09 07:55:46,427 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] regionserver.HStore(310): Store=fde6fda2ba4dd6622e7e423c1f8b61dd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,427 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,429 DEBUG [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2 2023-04-09 07:55:46,429 DEBUG [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2 2023-04-09 07:55:46,429 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fde6fda2ba4dd6622e7e423c1f8b61dd columnFamilyName family2 2023-04-09 07:55:46,430 INFO [StoreOpener-fde6fda2ba4dd6622e7e423c1f8b61dd-1] regionserver.HStore(310): Store=fde6fda2ba4dd6622e7e423c1f8b61dd/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-09 07:55:46,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,434 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-09 07:55:46,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:55:46,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-09 07:55:46,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fde6fda2ba4dd6622e7e423c1f8b61dd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64574081, jitterRate=-0.037771210074424744}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-09 07:55:46,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:46,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd., pid=121, masterSystemTime=1681026946327 2023-04-09 07:55:46,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:46,439 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=fde6fda2ba4dd6622e7e423c1f8b61dd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:46,439 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1681026946439"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1681026946439"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1681026946439"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1681026946439"}]},"ts":"1681026946439"} 2023-04-09 07:55:46,443 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-04-09 07:55:46,443 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; OpenRegionProcedure fde6fda2ba4dd6622e7e423c1f8b61dd, server=jenkins-hbase4.apache.org,34907,1681026816245 in 265 msec 2023-04-09 07:55:46,444 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-04-09 07:55:46,444 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fde6fda2ba4dd6622e7e423c1f8b61dd, ASSIGN in 422 msec 2023-04-09 07:55:46,445 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-09 07:55:46,445 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1681026946445"}]},"ts":"1681026946445"} 2023-04-09 07:55:46,446 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-04-09 07:55:46,448 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-04-09 07:55:46,449 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2570 sec 2023-04-09 07:55:47,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-04-09 07:55:47,299 INFO [Listener at localhost/35977] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-04-09 07:55:47,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:47,306 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:47,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:47,309 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:47,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-09 07:55:47,311 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-09 07:55:47,311 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:47,311 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-09 07:55:47,311 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-09 07:55:47,312 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:47,312 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:47,315 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:47,315 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:47,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:47,315 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:47,315 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:47,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:47,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:47,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:47,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-09 07:55:47,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-09 07:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-09 07:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:47,317 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-09 07:55:47,317 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:47,317 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. started... 2023-04-09 07:55:47,317 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:47,317 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. started... 2023-04-09 07:55:47,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:47,318 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ce3a8bff0d6396ea7236f773f152c0ec 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:47,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:47,318 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a09e145f7e57c7d9f5ff2cc3dd5d6901 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:47,318 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:47,318 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. started... 2023-04-09 07:55:47,319 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fde6fda2ba4dd6622e7e423c1f8b61dd 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-04-09 07:55:47,403 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409d87a5b482ea54912a0492cce11b3451e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409d87a5b482ea54912a0492cce11b3451e 2023-04-09 07:55:47,406 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409706258b7b9cc46f3897bc024f4ea3e3d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409706258b7b9cc46f3897bc024f4ea3e3d 2023-04-09 07:55:47,429 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/9aef7c7333f54aa5acd2d31fcae34604 2023-04-09 07:55:47,508 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040925bf27aa2b7e41fdbad045c6252cd2c1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023040925bf27aa2b7e41fdbad045c6252cd2c1 2023-04-09 07:55:47,528 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/b69a5b35787448fa827910f4be20eb73 2023-04-09 07:55:47,533 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/9aef7c7333f54aa5acd2d31fcae34604 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604 2023-04-09 07:55:47,567 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604, entries=4, sequenceid=5, filesize=5.5 K 2023-04-09 07:55:47,568 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/b69a5b35787448fa827910f4be20eb73 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73 2023-04-09 07:55:47,576 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:47,577 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fde6fda2ba4dd6622e7e423c1f8b61dd in 258ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:47,578 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-04-09 07:55:47,578 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:47,578 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:47,578 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:47,791 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304096723b3cfe08940e7ae58636f0f56304d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304096723b3cfe08940e7ae58636f0f56304d 2023-04-09 07:55:47,812 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/33b639282c9f4ff0b09ac681f35d1c2a 2023-04-09 07:55:47,830 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/ce361ccf19fe4df7a10286424f2a6f0b 2023-04-09 07:55:47,868 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304091390070dbfb540009c62a043ca1b4320 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202304091390070dbfb540009c62a043ca1b4320 2023-04-09 07:55:47,883 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409a09253aff51942eb8299107180ab6e54 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230409a09253aff51942eb8299107180ab6e54 2023-04-09 07:55:47,885 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/888a2f56e08e48c1b59e3077012c71d5 2023-04-09 07:55:47,889 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/33b639282c9f4ff0b09ac681f35d1c2a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a 2023-04-09 07:55:47,926 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a, entries=4, sequenceid=5, filesize=5.5 K 2023-04-09 07:55:47,927 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/888a2f56e08e48c1b59e3077012c71d5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5 2023-04-09 07:55:47,930 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:47,931 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a09e145f7e57c7d9f5ff2cc3dd5d6901 in 613ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:47,933 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:47,933 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:47,933 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:48,304 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/d8cebef9b873495d8d53e59d0dec4899 2023-04-09 07:55:48,309 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/ce361ccf19fe4df7a10286424f2a6f0b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b 2023-04-09 07:55:48,347 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b, entries=4, sequenceid=5, filesize=5.5 K 2023-04-09 07:55:48,348 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/d8cebef9b873495d8d53e59d0dec4899 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899 2023-04-09 07:55:48,352 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899, entries=2, sequenceid=5, filesize=5.1 K 2023-04-09 07:55:48,354 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ce3a8bff0d6396ea7236f773f152c0ec in 1036ms, sequenceid=5, compaction requested=false 2023-04-09 07:55:48,354 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:48,354 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:48,354 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-09 07:55:48,355 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-09 07:55:48,360 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,360 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:48,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-09 07:55:48,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@aac4c11[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:48,362 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-09 07:55:48,362 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,363 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-09 07:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-09 07:55:48,363 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:48,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,364 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-09 07:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:48,365 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,365 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-09 07:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:48,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:48,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:48,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-09 07:55:48,367 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-09 07:55:48,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:48,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:48,367 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-09 07:55:48,367 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:48,370 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,370 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:48,370 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:48,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:48,371 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,372 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,374 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,374 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:48,374 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:48,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:48,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:48,375 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-09 07:55:48,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,375 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:48,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-09 07:55:48,376 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:48,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:48,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:48,476 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:48,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:48,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:48,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:48,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:48,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-09 07:55:48,483 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-09 07:55:48,483 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:48,483 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-09 07:55:48,483 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-09 07:55:48,484 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,484 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:48,485 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:48,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:48,485 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:48,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:48,486 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:48,486 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:48,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:48,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:48,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:48,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-09 07:55:48,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-09 07:55:48,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-09 07:55:48,489 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-09 07:55:48,489 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:48,489 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-09 07:55:48,489 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:48,489 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. started... 2023-04-09 07:55:48,489 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a09e145f7e57c7d9f5ff2cc3dd5d6901 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:48,490 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:48,490 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. started... 2023-04-09 07:55:48,490 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ce3a8bff0d6396ea7236f773f152c0ec 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:48,490 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:48,491 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:48,491 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:48,491 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. started... 2023-04-09 07:55:48,491 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fde6fda2ba4dd6622e7e423c1f8b61dd 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-04-09 07:55:48,560 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409de549f3688d7440c9aa9f6b12acdadb3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409de549f3688d7440c9aa9f6b12acdadb3 2023-04-09 07:55:48,972 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304099a91bedd80b84c6481b1bebf2a1bd0e2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099a91bedd80b84c6481b1bebf2a1bd0e2 2023-04-09 07:55:48,973 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040950058a6cebee416b8d6c70197a9a4965 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040950058a6cebee416b8d6c70197a9a4965 2023-04-09 07:55:48,981 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/d054f0020bac4977893f49a9b5af4b92 2023-04-09 07:55:49,069 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304097e91739cac6641b98707ce06f79a0f59 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202304097e91739cac6641b98707ce06f79a0f59 2023-04-09 07:55:49,079 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/6516f8aa0cf74217a89543eb23a57934 2023-04-09 07:55:49,085 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/d054f0020bac4977893f49a9b5af4b92 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92 2023-04-09 07:55:49,118 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92, entries=4, sequenceid=9, filesize=5.5 K 2023-04-09 07:55:49,119 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/6516f8aa0cf74217a89543eb23a57934 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934 2023-04-09 07:55:49,125 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:49,128 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fde6fda2ba4dd6622e7e423c1f8b61dd in 637ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:49,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:49,128 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:49,128 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:49,442 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/f21acfadbff64cc6abb14b27a2e43460 2023-04-09 07:55:49,451 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/87b4630d10924d0499f08edf74aefae6 2023-04-09 07:55:49,557 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409372b3fff68e34724896572aa0129a244 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230409372b3fff68e34724896572aa0129a244 2023-04-09 07:55:49,570 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/5feac2a9484a40338911c7994615a3b1 2023-04-09 07:55:49,577 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/87b4630d10924d0499f08edf74aefae6 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6 2023-04-09 07:55:49,613 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6, entries=4, sequenceid=9, filesize=5.5 K 2023-04-09 07:55:49,615 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/5feac2a9484a40338911c7994615a3b1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1 2023-04-09 07:55:49,620 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:49,621 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ce3a8bff0d6396ea7236f773f152c0ec in 1131ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:49,621 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:49,621 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:49,621 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:49,942 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202304096263ecb77bfa4cabad0fd0ede20dac66 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202304096263ecb77bfa4cabad0fd0ede20dac66 2023-04-09 07:55:50,354 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/482db45d30f0417fa92dfe6fea88bc75 2023-04-09 07:55:50,358 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/f21acfadbff64cc6abb14b27a2e43460 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460 2023-04-09 07:55:50,406 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460, entries=4, sequenceid=9, filesize=5.5 K 2023-04-09 07:55:50,407 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/482db45d30f0417fa92dfe6fea88bc75 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75 2023-04-09 07:55:50,423 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75, entries=2, sequenceid=9, filesize=5.1 K 2023-04-09 07:55:50,424 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a09e145f7e57c7d9f5ff2cc3dd5d6901 in 1935ms, sequenceid=9, compaction requested=false 2023-04-09 07:55:50,424 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:50,424 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:50,424 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:50,424 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:50,424 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,425 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-09 07:55:50,425 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-09 07:55:50,426 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,426 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:50,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:50,427 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:50,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:50,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,427 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:50,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:50,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-09 07:55:50,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6951157b[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:50,429 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-09 07:55:50,429 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,430 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-09 07:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-09 07:55:50,430 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:50,430 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,430 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:50,432 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-09 07:55:50,432 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:50,432 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,432 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:50,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:50,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:50,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-09 07:55:50,435 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-09 07:55:50,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:50,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:50,435 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-09 07:55:50,435 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:50,437 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,437 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,437 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:50,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:50,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:50,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,438 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:50,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,445 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,448 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:50,448 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:50,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:50,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:50,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:50,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:50,449 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,449 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:50,449 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:50,449 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-09 07:55:50,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:50,449 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:50,449 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,450 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,450 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:50,450 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,450 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:50,450 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:50,551 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:50,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:50,553 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:50,553 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:50,553 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:50,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:50,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:50,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-09 07:55:50,577 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-09 07:55:50,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:50,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-09 07:55:50,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-09 07:55:50,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,577 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:50,578 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:50,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:50,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:50,579 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:50,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:50,579 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:50,580 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:50,580 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-09 07:55:50,580 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-09 07:55:50,580 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-09 07:55:50,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-09 07:55:50,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:50,581 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-09 07:55:50,581 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:50,581 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. started... 2023-04-09 07:55:50,582 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a09e145f7e57c7d9f5ff2cc3dd5d6901 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:50,588 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:50,588 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. started... 2023-04-09 07:55:50,588 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ce3a8bff0d6396ea7236f773f152c0ec 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:50,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:50,592 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:50,592 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:50,592 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. started... 2023-04-09 07:55:50,593 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fde6fda2ba4dd6622e7e423c1f8b61dd 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-04-09 07:55:50,694 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409ce2c060ffd81467fb8dbdca37092f499 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce2c060ffd81467fb8dbdca37092f499 2023-04-09 07:55:50,716 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409c26f39c573834708ac4132aa12216768 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409c26f39c573834708ac4132aa12216768 2023-04-09 07:55:50,754 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/4b0a0f31afac46c880608464fc81b810 2023-04-09 07:55:51,093 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409b34666ee1ee04a4590f6204927086997 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409b34666ee1ee04a4590f6204927086997 2023-04-09 07:55:51,134 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/fccf51cb2aef4c7a83836eb85818474d 2023-04-09 07:55:51,178 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230409e87d20e00bd84e9eb097d0be96b0b5c4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230409e87d20e00bd84e9eb097d0be96b0b5c4 2023-04-09 07:55:51,192 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/ff03860882d14efe8bd16c6d9f5444e5 2023-04-09 07:55:51,198 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/fccf51cb2aef4c7a83836eb85818474d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d 2023-04-09 07:55:51,214 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304099a32fc6616394c27a00d7d3de910dc5b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202304099a32fc6616394c27a00d7d3de910dc5b 2023-04-09 07:55:51,230 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/89aca5be359f498581f3caf7ca09be3c 2023-04-09 07:55:51,235 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/4b0a0f31afac46c880608464fc81b810 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810 2023-04-09 07:55:51,243 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d, entries=4, sequenceid=13, filesize=5.5 K 2023-04-09 07:55:51,244 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/ff03860882d14efe8bd16c6d9f5444e5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5 2023-04-09 07:55:51,252 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:51,253 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fde6fda2ba4dd6622e7e423c1f8b61dd in 661ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:51,253 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:51,253 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:51,253 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:51,275 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810, entries=4, sequenceid=13, filesize=5.5 K 2023-04-09 07:55:51,276 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/89aca5be359f498581f3caf7ca09be3c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c 2023-04-09 07:55:51,280 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:51,283 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ce3a8bff0d6396ea7236f773f152c0ec in 695ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:51,284 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:51,284 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:51,284 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:51,514 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/95cf935a6f064393a395dd0c762f8fd5 2023-04-09 07:55:51,556 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040967562c27dd044d3a9acf05739d5a25cc to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023040967562c27dd044d3a9acf05739d5a25cc 2023-04-09 07:55:51,561 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/5b0b2d010f004ada97f105d46649f9f2 2023-04-09 07:55:51,564 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/95cf935a6f064393a395dd0c762f8fd5 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5 2023-04-09 07:55:51,592 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5, entries=4, sequenceid=13, filesize=5.5 K 2023-04-09 07:55:51,593 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/5b0b2d010f004ada97f105d46649f9f2 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2 2023-04-09 07:55:51,597 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2, entries=2, sequenceid=13, filesize=5.1 K 2023-04-09 07:55:51,598 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a09e145f7e57c7d9f5ff2cc3dd5d6901 in 1017ms, sequenceid=13, compaction requested=true 2023-04-09 07:55:51,598 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:51,598 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:51,598 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:51,598 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:51,598 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-09 07:55:51,599 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-09 07:55:51,600 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,601 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:51,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:51,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-09 07:55:51,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c48c155[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:51,602 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-09 07:55:51,602 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,603 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-09 07:55:51,603 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,604 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:51,603 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-09 07:55:51,604 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:51,605 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-09 07:55:51,605 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,605 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:51,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,605 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-09 07:55:51,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:51,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:51,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:51,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:51,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:51,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-09 07:55:51,607 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-09 07:55:51,607 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:51,607 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:51,607 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-09 07:55:51,607 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:51,610 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,610 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,610 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:51,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:51,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:51,611 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:51,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,612 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:51,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:51,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:51,614 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:51,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:51,614 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-09 07:55:51,615 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:51,614 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:51,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:51,615 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,615 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:51,615 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:51,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:51,715 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:51,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:51,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:51,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:51,717 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. with WAL disabled. Data may be lost in the event of a crash. 2023-04-09 07:55:51,720 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:51,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:51,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-09 07:55:51,721 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-09 07:55:51,722 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:51,722 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-09 07:55:51,722 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-09 07:55:51,722 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,722 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:51,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,726 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:51,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:51,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:51,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:51,726 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:51,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:51,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:51,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:51,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-09 07:55:51,728 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-09 07:55:51,728 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-09 07:55:51,732 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-09 07:55:51,732 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:51,732 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:51,736 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:51,736 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. started... 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. started... 2023-04-09 07:55:51,736 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. started... 2023-04-09 07:55:51,736 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fde6fda2ba4dd6622e7e423c1f8b61dd 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:51,736 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ce3a8bff0d6396ea7236f773f152c0ec 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:51,736 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a09e145f7e57c7d9f5ff2cc3dd5d6901 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-04-09 07:55:51,810 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304099629cbe5def04d6593487eeea188806f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099629cbe5def04d6593487eeea188806f 2023-04-09 07:55:51,811 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304096b6223d5f72b402baa62ed6d206d50b4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304096b6223d5f72b402baa62ed6d206d50b4 2023-04-09 07:55:51,832 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-09 07:55:52,213 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040938e1d3b57c194eea823d2440b39c12b9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040938e1d3b57c194eea823d2440b39c12b9 2023-04-09 07:55:52,232 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/88ab7582b3f64240b0779b51354a9442 2023-04-09 07:55:52,236 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/bef5ab615aa242078f02e179a3dc630b 2023-04-09 07:55:52,241 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/02cf488b6937437c8d211b8f167963b4 2023-04-09 07:55:52,280 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304098672760462be49bcbe02f41f517fd054 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202304098672760462be49bcbe02f41f517fd054 2023-04-09 07:55:52,306 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/f96b6a9018f74481827d643d1b8cd341 2023-04-09 07:55:52,308 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409212b7de845494c67b3eea6ebd5e7232d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230409212b7de845494c67b3eea6ebd5e7232d 2023-04-09 07:55:52,311 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/bef5ab615aa242078f02e179a3dc630b as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b 2023-04-09 07:55:52,322 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/001025f665a74411b336c3df98599f68 2023-04-09 07:55:52,333 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/02cf488b6937437c8d211b8f167963b4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4 2023-04-09 07:55:52,342 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b, entries=4, sequenceid=17, filesize=5.5 K 2023-04-09 07:55:52,343 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/f96b6a9018f74481827d643d1b8cd341 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341 2023-04-09 07:55:52,351 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:52,351 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fde6fda2ba4dd6622e7e423c1f8b61dd in 615ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:52,352 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:52,352 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:52,352 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:52,361 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4, entries=4, sequenceid=17, filesize=5.5 K 2023-04-09 07:55:52,363 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/001025f665a74411b336c3df98599f68 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68 2023-04-09 07:55:52,369 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:52,371 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ce3a8bff0d6396ea7236f773f152c0ec in 635ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:52,371 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:52,371 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:52,371 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:52,677 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409ec3b168ccdbf4636b69fce99cfe0eb18 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230409ec3b168ccdbf4636b69fce99cfe0eb18 2023-04-09 07:55:52,683 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/debf4b2b4dc14e449dc2fc9209efa093 2023-04-09 07:55:52,686 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/88ab7582b3f64240b0779b51354a9442 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442 2023-04-09 07:55:52,713 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442, entries=4, sequenceid=17, filesize=5.5 K 2023-04-09 07:55:52,714 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/debf4b2b4dc14e449dc2fc9209efa093 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093 2023-04-09 07:55:52,719 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093, entries=2, sequenceid=17, filesize=5.1 K 2023-04-09 07:55:52,720 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a09e145f7e57c7d9f5ff2cc3dd5d6901 in 984ms, sequenceid=17, compaction requested=true 2023-04-09 07:55:52,720 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:52,720 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:52,720 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:52,720 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:52,720 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,720 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-09 07:55:52,720 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-09 07:55:52,722 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,722 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:52,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:52,722 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,722 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:52,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:52,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:52,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:52,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-09 07:55:52,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20bae2b3[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:52,725 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-09 07:55:52,725 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,728 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-09 07:55:52,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-09 07:55:52,728 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:52,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,728 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:52,729 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,729 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-09 07:55:52,729 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:52,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:52,729 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-09 07:55:52,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:52,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:52,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:52,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:52,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-09 07:55:52,733 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-09 07:55:52,733 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:52,733 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:52,733 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-09 07:55:52,733 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:52,735 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,735 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,735 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:52,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:52,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:52,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:52,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:52,740 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:52,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,742 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:52,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,744 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:52,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-09 07:55:52,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:52,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:52,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:52,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:52,744 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:52,845 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:52,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:52,856 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:52,864 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:52,872 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:52,872 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:52,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-04-09 07:55:52,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-04-09 07:55:52,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-04-09 07:55:52,963 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-04-09 07:55:52,963 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:52,963 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-04-09 07:55:52,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-04-09 07:55:52,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,964 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:52,966 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:52,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:52,966 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:52,967 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:52,967 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-04-09 07:55:52,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-04-09 07:55:52,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-04-09 07:55:52,968 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-04-09 07:55:52,968 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-09 07:55:52,968 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-04-09 07:55:52,968 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:52,968 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. started... 2023-04-09 07:55:52,968 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a09e145f7e57c7d9f5ff2cc3dd5d6901 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:52,974 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:52,974 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-04-09 07:55:52,974 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-09 07:55:52,974 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. started... 2023-04-09 07:55:52,975 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:52,975 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ce3a8bff0d6396ea7236f773f152c0ec 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:52,975 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. started... 2023-04-09 07:55:52,975 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing fde6fda2ba4dd6622e7e423c1f8b61dd 2/2 column families, dataSize=138 B heapSize=944 B 2023-04-09 07:55:53,041 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/bd6384da0a5b4941bf772da601dbea39 2023-04-09 07:55:53,047 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/93666e05aba843dd8b82210a962e7336 2023-04-09 07:55:53,072 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93666e05aba843dd8b82210a962e7336 2023-04-09 07:55:53,075 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd6384da0a5b4941bf772da601dbea39 2023-04-09 07:55:53,502 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/ea68303df18e4976904a6eb2d4857f63 2023-04-09 07:55:53,503 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/3967f1ea108d4986b481b3943be0e009 2023-04-09 07:55:53,508 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea68303df18e4976904a6eb2d4857f63 2023-04-09 07:55:53,508 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3967f1ea108d4986b481b3943be0e009 2023-04-09 07:55:53,508 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/93666e05aba843dd8b82210a962e7336 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336 2023-04-09 07:55:53,508 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/bd6384da0a5b4941bf772da601dbea39 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39 2023-04-09 07:55:53,542 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd6384da0a5b4941bf772da601dbea39 2023-04-09 07:55:53,542 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39, entries=3, sequenceid=23, filesize=5.2 K 2023-04-09 07:55:53,542 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93666e05aba843dd8b82210a962e7336 2023-04-09 07:55:53,542 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336, entries=3, sequenceid=23, filesize=5.2 K 2023-04-09 07:55:53,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/3967f1ea108d4986b481b3943be0e009 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009 2023-04-09 07:55:53,548 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/ea68303df18e4976904a6eb2d4857f63 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63 2023-04-09 07:55:53,564 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea68303df18e4976904a6eb2d4857f63 2023-04-09 07:55:53,565 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:53,565 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fde6fda2ba4dd6622e7e423c1f8b61dd in 590ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:53,566 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:53,566 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:53,566 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-04-09 07:55:53,568 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3967f1ea108d4986b481b3943be0e009 2023-04-09 07:55:53,568 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:53,569 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a09e145f7e57c7d9f5ff2cc3dd5d6901 in 601ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:53,569 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:53,569 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:53,570 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-04-09 07:55:53,829 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/b5b22487999d4604ac546fea0738baa2 2023-04-09 07:55:53,862 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5b22487999d4604ac546fea0738baa2 2023-04-09 07:55:54,275 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/da6137d8dde14364bb5b59c8ef89670a 2023-04-09 07:55:54,278 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da6137d8dde14364bb5b59c8ef89670a 2023-04-09 07:55:54,279 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/b5b22487999d4604ac546fea0738baa2 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2 2023-04-09 07:55:54,312 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b5b22487999d4604ac546fea0738baa2 2023-04-09 07:55:54,312 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2, entries=3, sequenceid=23, filesize=5.2 K 2023-04-09 07:55:54,313 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/da6137d8dde14364bb5b59c8ef89670a as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a 2023-04-09 07:55:54,317 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da6137d8dde14364bb5b59c8ef89670a 2023-04-09 07:55:54,317 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a, entries=1, sequenceid=23, filesize=5.0 K 2023-04-09 07:55:54,318 INFO [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ce3a8bff0d6396ea7236f773f152c0ec in 1343ms, sequenceid=23, compaction requested=true 2023-04-09 07:55:54,318 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:54,318 DEBUG [rs(jenkins-hbase4.apache.org,34907,1681026816245)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:54,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-04-09 07:55:54,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-04-09 07:55:54,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-04-09 07:55:54,318 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-04-09 07:55:54,325 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,325 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,326 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-09 07:55:54,328 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:54,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34907,1681026816245' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-04-09 07:55:54,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@489eee71[Count = 0] remaining members to acquire global barrier 2023-04-09 07:55:54,330 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-04-09 07:55:54,330 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,332 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-04-09 07:55:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-04-09 07:55:54,332 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,34907,1681026816245' in zk 2023-04-09 07:55:54,332 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,332 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-09 07:55:54,333 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-04-09 07:55:54,333 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:54,333 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,333 DEBUG [member: 'jenkins-hbase4.apache.org,34907,1681026816245' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-04-09 07:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:54,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,34907,1681026816245': 2023-04-09 07:55:54,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34907,1681026816245' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-04-09 07:55:54,336 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-04-09 07:55:54,336 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-09 07:55:54,336 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-09 07:55:54,336 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-04-09 07:55:54,336 INFO [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-09 07:55:54,338 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,338 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:54,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,339 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-09 07:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-09 07:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-09 07:55:54,339 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-09 07:55:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,340 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-09 07:55:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-04-09 07:55:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,343 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-09 07:55:54,343 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-09 07:55:54,343 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-04-09 07:55:54,343 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-09 07:55:54,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-09 07:55:54,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-09 07:55:54,343 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-04-09 07:55:54,343 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-04-09 07:55:54,344 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-04-09 07:55:54,343 DEBUG [(jenkins-hbase4.apache.org,44231,1681026814996)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-09 07:55:54,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-09 07:55:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-09 07:55:54,444 DEBUG [Listener at localhost/35977] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-09 07:55:54,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44231] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-09 07:55:54,445 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:54,445 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 is initiating major compaction (all files) 2023-04-09 07:55:54,445 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:54,445 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:54,447 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 in testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:54,447 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp, totalSize=27.1 K 2023-04-09 07:55:54,448 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 33b639282c9f4ff0b09ac681f35d1c2a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947306 2023-04-09 07:55:54,448 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting f21acfadbff64cc6abb14b27a2e43460, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:54,449 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 95cf935a6f064393a395dd0c762f8fd5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:54,449 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 88ab7582b3f64240b0779b51354a9442, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:54,449 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting bd6384da0a5b4941bf772da601dbea39, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:54,461 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): a09e145f7e57c7d9f5ff2cc3dd5d6901#family1#compaction#197 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:54,511 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del 2023-04-09 07:55:54,512 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del 2023-04-09 07:55:55,000 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:55:55,000 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0471502c to 127.0.0.1:57867 2023-04-09 07:55:55,000 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:55:55,015 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d36d39e585614370aaef778281d7ce7c 2023-04-09 07:55:55,016 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family1/d36d39e585614370aaef778281d7ce7c as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/d36d39e585614370aaef778281d7ce7c 2023-04-09 07:55:55,053 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d36d39e585614370aaef778281d7ce7c 2023-04-09 07:55:55,054 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 of a09e145f7e57c7d9f5ff2cc3dd5d6901 into d36d39e585614370aaef778281d7ce7c(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:55,054 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:55,054 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:55,054 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): a09e145f7e57c7d9f5ff2cc3dd5d6901/family2 is initiating major compaction (all files) 2023-04-09 07:55:55,054 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:55,054 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:55,055 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of a09e145f7e57c7d9f5ff2cc3dd5d6901/family2 in testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:55:55,055 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp, totalSize=25.4 K 2023-04-09 07:55:55,055 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 888a2f56e08e48c1b59e3077012c71d5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947306 2023-04-09 07:55:55,056 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 482db45d30f0417fa92dfe6fea88bc75, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:55,057 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 5b0b2d010f004ada97f105d46649f9f2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:55,057 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting debf4b2b4dc14e449dc2fc9209efa093, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:55,057 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 3967f1ea108d4986b481b3943be0e009, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:55,075 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): a09e145f7e57c7d9f5ff2cc3dd5d6901#family2#compaction#198 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:55,083 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023040979dc17996d1244b9a654b32c81cbdbb5_del 2023-04-09 07:55:55,083 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023040979dc17996d1244b9a654b32c81cbdbb5_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023040979dc17996d1244b9a654b32c81cbdbb5_del 2023-04-09 07:55:55,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39] to archive 2023-04-09 07:55:55,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:55,107 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6879690d64c24aeb971cd8d930f03915 2023-04-09 07:55:55,108 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/.tmp/family2/6879690d64c24aeb971cd8d930f03915 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/6879690d64c24aeb971cd8d930f03915 2023-04-09 07:55:55,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/33b639282c9f4ff0b09ac681f35d1c2a 2023-04-09 07:55:55,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/f21acfadbff64cc6abb14b27a2e43460 2023-04-09 07:55:55,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/95cf935a6f064393a395dd0c762f8fd5 2023-04-09 07:55:55,112 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6879690d64c24aeb971cd8d930f03915 2023-04-09 07:55:55,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/88ab7582b3f64240b0779b51354a9442 2023-04-09 07:55:55,114 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a09e145f7e57c7d9f5ff2cc3dd5d6901/family2 of a09e145f7e57c7d9f5ff2cc3dd5d6901 into 6879690d64c24aeb971cd8d930f03915(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:55,114 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:55:55,114 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:55,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/bd6384da0a5b4941bf772da601dbea39 2023-04-09 07:55:55,114 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): ce3a8bff0d6396ea7236f773f152c0ec/family1 is initiating major compaction (all files) 2023-04-09 07:55:55,115 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:55,115 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:55,115 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of ce3a8bff0d6396ea7236f773f152c0ec/family1 in testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:55,115 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp, totalSize=27.1 K 2023-04-09 07:55:55,115 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting ce361ccf19fe4df7a10286424f2a6f0b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947306 2023-04-09 07:55:55,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009] to archive 2023-04-09 07:55:55,116 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 87b4630d10924d0499f08edf74aefae6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:55,116 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 4b0a0f31afac46c880608464fc81b810, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:55,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:55,116 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 02cf488b6937437c8d211b8f167963b4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:55,117 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting b5b22487999d4604ac546fea0738baa2, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:55,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/888a2f56e08e48c1b59e3077012c71d5 2023-04-09 07:55:55,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/482db45d30f0417fa92dfe6fea88bc75 2023-04-09 07:55:55,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/5b0b2d010f004ada97f105d46649f9f2 2023-04-09 07:55:55,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/debf4b2b4dc14e449dc2fc9209efa093 2023-04-09 07:55:55,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family2/3967f1ea108d4986b481b3943be0e009 2023-04-09 07:55:55,126 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): ce3a8bff0d6396ea7236f773f152c0ec#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:55,577 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del 2023-04-09 07:55:55,577 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del 2023-04-09 07:55:56,039 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0059d5461bfc4f889d407d385a58c7b4 2023-04-09 07:55:56,040 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family1/0059d5461bfc4f889d407d385a58c7b4 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/0059d5461bfc4f889d407d385a58c7b4 2023-04-09 07:55:56,067 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0059d5461bfc4f889d407d385a58c7b4 2023-04-09 07:55:56,068 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ce3a8bff0d6396ea7236f773f152c0ec/family1 of ce3a8bff0d6396ea7236f773f152c0ec into 0059d5461bfc4f889d407d385a58c7b4(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:56,068 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:56,069 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:56,069 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): ce3a8bff0d6396ea7236f773f152c0ec/family2 is initiating major compaction (all files) 2023-04-09 07:55:56,069 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:56,069 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:56,069 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of ce3a8bff0d6396ea7236f773f152c0ec/family2 in testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:55:56,069 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp, totalSize=25.4 K 2023-04-09 07:55:56,070 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting d8cebef9b873495d8d53e59d0dec4899, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947306 2023-04-09 07:55:56,070 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 5feac2a9484a40338911c7994615a3b1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:56,071 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 89aca5be359f498581f3caf7ca09be3c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:56,071 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 001025f665a74411b336c3df98599f68, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:56,072 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting da6137d8dde14364bb5b59c8ef89670a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:56,081 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): ce3a8bff0d6396ea7236f773f152c0ec#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:56,093 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230409ddc8fe8d7c5b44f8b78b9f5f0cd8bafa_del 2023-04-09 07:55:56,093 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230409ddc8fe8d7c5b44f8b78b9f5f0cd8bafa_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230409ddc8fe8d7c5b44f8b78b9f5f0cd8bafa_del 2023-04-09 07:55:56,104 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8176ae15cf9044a1b47b1f2b7328b950 2023-04-09 07:55:56,105 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/.tmp/family2/8176ae15cf9044a1b47b1f2b7328b950 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/8176ae15cf9044a1b47b1f2b7328b950 2023-04-09 07:55:56,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2] to archive 2023-04-09 07:55:56,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:56,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/ce361ccf19fe4df7a10286424f2a6f0b 2023-04-09 07:55:56,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/87b4630d10924d0499f08edf74aefae6 2023-04-09 07:55:56,109 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8176ae15cf9044a1b47b1f2b7328b950 2023-04-09 07:55:56,110 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ce3a8bff0d6396ea7236f773f152c0ec/family2 of ce3a8bff0d6396ea7236f773f152c0ec into 8176ae15cf9044a1b47b1f2b7328b950(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:56,110 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:56,110 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:56,110 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): fde6fda2ba4dd6622e7e423c1f8b61dd/family1 is initiating major compaction (all files) 2023-04-09 07:55:56,110 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:56,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/4b0a0f31afac46c880608464fc81b810 2023-04-09 07:55:56,110 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:56,111 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of fde6fda2ba4dd6622e7e423c1f8b61dd/family1 in testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:56,111 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp, totalSize=27.1 K 2023-04-09 07:55:56,112 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 9aef7c7333f54aa5acd2d31fcae34604, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947307 2023-04-09 07:55:56,112 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting d054f0020bac4977893f49a9b5af4b92, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:56,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/02cf488b6937437c8d211b8f167963b4 2023-04-09 07:55:56,113 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting fccf51cb2aef4c7a83836eb85818474d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:56,113 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting bef5ab615aa242078f02e179a3dc630b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:56,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/b5b22487999d4604ac546fea0738baa2 2023-04-09 07:55:56,114 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 93666e05aba843dd8b82210a962e7336, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:56,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a] to archive 2023-04-09 07:55:56,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:56,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/d8cebef9b873495d8d53e59d0dec4899 2023-04-09 07:55:56,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/5feac2a9484a40338911c7994615a3b1 2023-04-09 07:55:56,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/89aca5be359f498581f3caf7ca09be3c 2023-04-09 07:55:56,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/001025f665a74411b336c3df98599f68 2023-04-09 07:55:56,124 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family2/da6137d8dde14364bb5b59c8ef89670a 2023-04-09 07:55:56,124 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): fde6fda2ba4dd6622e7e423c1f8b61dd#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:56,171 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del 2023-04-09 07:55:56,172 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del 2023-04-09 07:55:56,633 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ed30317127b423ab2f02001ffaf74cf 2023-04-09 07:55:56,633 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family1/7ed30317127b423ab2f02001ffaf74cf as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/7ed30317127b423ab2f02001ffaf74cf 2023-04-09 07:55:56,659 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ed30317127b423ab2f02001ffaf74cf 2023-04-09 07:55:56,660 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fde6fda2ba4dd6622e7e423c1f8b61dd/family1 of fde6fda2ba4dd6622e7e423c1f8b61dd into 7ed30317127b423ab2f02001ffaf74cf(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:56,660 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:56,661 DEBUG [Listener at localhost/35977] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-04-09 07:55:56,661 DEBUG [Listener at localhost/35977] regionserver.HStore(1912): fde6fda2ba4dd6622e7e423c1f8b61dd/family2 is initiating major compaction (all files) 2023-04-09 07:55:56,661 INFO [Listener at localhost/35977] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-09 07:55:56,661 INFO [Listener at localhost/35977] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-09 07:55:56,661 INFO [Listener at localhost/35977] regionserver.HRegion(2259): Starting compaction of fde6fda2ba4dd6622e7e423c1f8b61dd/family2 in testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:55:56,661 INFO [Listener at localhost/35977] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63] into tmpdir=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp, totalSize=25.4 K 2023-04-09 07:55:56,662 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting b69a5b35787448fa827910f4be20eb73, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1681026947307 2023-04-09 07:55:56,662 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting 6516f8aa0cf74217a89543eb23a57934, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1681026948478 2023-04-09 07:55:56,663 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting ff03860882d14efe8bd16c6d9f5444e5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1681026950553 2023-04-09 07:55:56,664 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting f96b6a9018f74481827d643d1b8cd341, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1681026951717 2023-04-09 07:55:56,664 DEBUG [Listener at localhost/35977] compactions.Compactor(207): Compacting ea68303df18e4976904a6eb2d4857f63, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-04-09 07:55:56,674 INFO [Listener at localhost/35977] throttle.PressureAwareThroughputController(145): fde6fda2ba4dd6622e7e423c1f8b61dd#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-09 07:55:56,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336] to archive 2023-04-09 07:55:56,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:56,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/9aef7c7333f54aa5acd2d31fcae34604 2023-04-09 07:55:56,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d054f0020bac4977893f49a9b5af4b92 2023-04-09 07:55:56,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/fccf51cb2aef4c7a83836eb85818474d 2023-04-09 07:55:56,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/bef5ab615aa242078f02e179a3dc630b 2023-04-09 07:55:56,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/93666e05aba843dd8b82210a962e7336 2023-04-09 07:55:57,085 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023040987b9fe470f0e4bb9868abd08404969f0_del 2023-04-09 07:55:57,085 INFO [Listener at localhost/35977] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023040987b9fe470f0e4bb9868abd08404969f0_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023040987b9fe470f0e4bb9868abd08404969f0_del 2023-04-09 07:55:57,497 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2247ca69308246e09e43150d2ff33aa7 2023-04-09 07:55:57,498 DEBUG [Listener at localhost/35977] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/.tmp/family2/2247ca69308246e09e43150d2ff33aa7 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/2247ca69308246e09e43150d2ff33aa7 2023-04-09 07:55:57,501 INFO [Listener at localhost/35977] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2247ca69308246e09e43150d2ff33aa7 2023-04-09 07:55:57,501 INFO [Listener at localhost/35977] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fde6fda2ba4dd6622e7e423c1f8b61dd/family2 of fde6fda2ba4dd6622e7e423c1f8b61dd into 2247ca69308246e09e43150d2ff33aa7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-09 07:55:57,502 DEBUG [Listener at localhost/35977] regionserver.HRegion(2289): Compaction status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:55:57,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63] to archive 2023-04-09 07:55:57,506 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:57,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:57,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/b69a5b35787448fa827910f4be20eb73 2023-04-09 07:55:57,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/6516f8aa0cf74217a89543eb23a57934 2023-04-09 07:55:57,509 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:57,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ff03860882d14efe8bd16c6d9f5444e5 2023-04-09 07:55:57,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/f96b6a9018f74481827d643d1b8cd341 2023-04-09 07:55:57,512 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:55:57,513 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:55:57,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family2/ea68303df18e4976904a6eb2d4857f63 2023-04-09 07:55:57,518 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44231] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10&5\x8ABC\xA4\x1EI\xF9\x18"\x07\xFE\xE6\x95\xF7"\x10'g\x98i{\xF6\xD5\xEF\x0Bg\xC4\xF2\x91P\x14\xB6*\x10X%\x16\x05\x90;4\xBD0d!\xBD\x92a\xCE\xF12\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-04-09 07:55:57,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-09 07:55:57,541 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-09 07:55:57,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:55:57,542 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1681027557542 2023-04-09 07:55:57,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-04-09 07:55:57,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-04-09 07:55:57,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del 2023-04-09 07:55:58,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del 2023-04-09 07:55:58,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del 2023-04-09 07:55:58,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:55:58,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] zookeeper.ReadOnlyZKClient(139): Connect 0x6fa50531 to 127.0.0.1:57867 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-09 07:55:58,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45a5be6e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-09 07:55:58,054 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:55:59,039 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202304093917ac41dd484552a3e08cdcc1be4147 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304093917ac41dd484552a3e08cdcc1be4147 2023-04-09 07:55:59,040 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:55:59,052 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-09 07:55:59,055 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58136, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-09 07:55:59,070 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-09 07:55:59,072 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33878, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-09 07:55:59,113 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230409/family1/7af30aca4a73411da107e53e0fea492d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-04-09 07:55:59,114 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230409/family1/7af30aca4a73411da107e53e0fea492d}] 2023-04-09 07:55:59,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230409/family1/7af30aca4a73411da107e53e0fea492d for inclusion in ce3a8bff0d6396ea7236f773f152c0ec/family1 2023-04-09 07:55:59,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-04-09 07:55:59,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=1 last=2 2023-04-09 07:55:59,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:55:59,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230409/family1/7af30aca4a73411da107e53e0fea492d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__4kd2i9r91fo1ijk2kkkq8r8p3oleh8kemq3cqr6obts0f2n6hjt73a0ccdo4joqa/family1/7af30aca4a73411da107e53e0fea492d 2023-04-09 07:55:59,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__4kd2i9r91fo1ijk2kkkq8r8p3oleh8kemq3cqr6obts0f2n6hjt73a0ccdo4joqa/family1/7af30aca4a73411da107e53e0fea492d as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/76f103d06ff045c8b74c42404c05b2fd_SeqId_28_ 2023-04-09 07:55:59,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__4kd2i9r91fo1ijk2kkkq8r8p3oleh8kemq3cqr6obts0f2n6hjt73a0ccdo4joqa/family1/7af30aca4a73411da107e53e0fea492d into ce3a8bff0d6396ea7236f773f152c0ec/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/76f103d06ff045c8b74c42404c05b2fd_SeqId_28_ - updating store file list. 2023-04-09 07:55:59,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/76f103d06ff045c8b74c42404c05b2fd_SeqId_28_ into ce3a8bff0d6396ea7236f773f152c0ec/family1 2023-04-09 07:55:59,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__4kd2i9r91fo1ijk2kkkq8r8p3oleh8kemq3cqr6obts0f2n6hjt73a0ccdo4joqa/family1/7af30aca4a73411da107e53e0fea492d into ce3a8bff0d6396ea7236f773f152c0ec/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/family1/76f103d06ff045c8b74c42404c05b2fd_SeqId_28_) 2023-04-09 07:55:59,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__4kd2i9r91fo1ijk2kkkq8r8p3oleh8kemq3cqr6obts0f2n6hjt73a0ccdo4joqa/family1/7af30aca4a73411da107e53e0fea492d 2023-04-09 07:55:59,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:55:59,181 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:55:59,182 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099629cbe5def04d6593487eeea188806f to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099629cbe5def04d6593487eeea188806f 2023-04-09 07:55:59,185 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099a91bedd80b84c6481b1bebf2a1bd0e2 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304099a91bedd80b84c6481b1bebf2a1bd0e2 2023-04-09 07:55:59,186 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409c26f39c573834708ac4132aa12216768 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409c26f39c573834708ac4132aa12216768 2023-04-09 07:55:59,187 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409d87a5b482ea54912a0492cce11b3451e to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230409d87a5b482ea54912a0492cce11b3451e 2023-04-09 07:55:59,187 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-09 07:55:59,187 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230409' 2023-04-09 07:55:59,187 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:55:59,187 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:56:00,189 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202304096b95d8f05ffb4de49e81a1d9f40bd491 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304096b95d8f05ffb4de49e81a1d9f40bd491 2023-04-09 07:56:00,192 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:56:00,246 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a02697379b554090a95d7002b3c57e94 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-04-09 07:56:00,247 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a02697379b554090a95d7002b3c57e94}] 2023-04-09 07:56:00,249 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a02697379b554090a95d7002b3c57e94 for inclusion in fde6fda2ba4dd6622e7e423c1f8b61dd/family1 2023-04-09 07:56:00,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-04-09 07:56:00,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first=2 last= 2023-04-09 07:56:00,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:56:00,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230409/family1/a02697379b554090a95d7002b3c57e94 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fcbt28g2095vi4f45uo4qsf7ik7nnn1emd2hg8ucfj9fhqj788i20er5udtv11ij/family1/a02697379b554090a95d7002b3c57e94 2023-04-09 07:56:00,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fcbt28g2095vi4f45uo4qsf7ik7nnn1emd2hg8ucfj9fhqj788i20er5udtv11ij/family1/a02697379b554090a95d7002b3c57e94 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d96f69787f1448aea4708827dd342d4e_SeqId_28_ 2023-04-09 07:56:00,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fcbt28g2095vi4f45uo4qsf7ik7nnn1emd2hg8ucfj9fhqj788i20er5udtv11ij/family1/a02697379b554090a95d7002b3c57e94 into fde6fda2ba4dd6622e7e423c1f8b61dd/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d96f69787f1448aea4708827dd342d4e_SeqId_28_ - updating store file list. 2023-04-09 07:56:00,297 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d96f69787f1448aea4708827dd342d4e_SeqId_28_ into fde6fda2ba4dd6622e7e423c1f8b61dd/family1 2023-04-09 07:56:00,297 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fcbt28g2095vi4f45uo4qsf7ik7nnn1emd2hg8ucfj9fhqj788i20er5udtv11ij/family1/a02697379b554090a95d7002b3c57e94 into fde6fda2ba4dd6622e7e423c1f8b61dd/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/family1/d96f69787f1448aea4708827dd342d4e_SeqId_28_) 2023-04-09 07:56:00,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fcbt28g2095vi4f45uo4qsf7ik7nnn1emd2hg8ucfj9fhqj788i20er5udtv11ij/family1/a02697379b554090a95d7002b3c57e94 2023-04-09 07:56:00,299 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:56:00,300 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:56:00,301 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304096b6223d5f72b402baa62ed6d206d50b4 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202304096b6223d5f72b402baa62ed6d206d50b4 2023-04-09 07:56:00,303 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409706258b7b9cc46f3897bc024f4ea3e3d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409706258b7b9cc46f3897bc024f4ea3e3d 2023-04-09 07:56:00,304 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce2c060ffd81467fb8dbdca37092f499 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409ce2c060ffd81467fb8dbdca37092f499 2023-04-09 07:56:00,304 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409de549f3688d7440c9aa9f6b12acdadb3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230409de549f3688d7440c9aa9f6b12acdadb3 2023-04-09 07:56:00,305 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-09 07:56:00,305 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230409' 2023-04-09 07:56:00,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:56:00,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:56:00,879 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230409e782e599abc4449c97e5880ecc13e9d3 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409e782e599abc4449c97e5880ecc13e9d3 2023-04-09 07:56:00,880 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:56:00,937 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230409/family1/59db221a30284323ac2b6553e811ec79 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-04-09 07:56:00,938 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901., hostname=jenkins-hbase4.apache.org,34907,1681026816245, seqNum=2 for row with hfile group [{family1,hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230409/family1/59db221a30284323ac2b6553e811ec79}] 2023-04-09 07:56:00,940 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(763): Validating hfile at hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230409/family1/59db221a30284323ac2b6553e811ec79 for inclusion in a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 2023-04-09 07:56:00,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-04-09 07:56:00,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(777): Region bounds: first= last=1 2023-04-09 07:56:00,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegion(2446): Flush status journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:56:00,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230409/family1/59db221a30284323ac2b6553e811ec79 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fa841uqcc1kq0430ittm5okaasqq009bt6gmhn02ob2ik1gibocnb0nti71ug0o/family1/59db221a30284323ac2b6553e811ec79 2023-04-09 07:56:00,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fa841uqcc1kq0430ittm5okaasqq009bt6gmhn02ob2ik1gibocnb0nti71ug0o/family1/59db221a30284323ac2b6553e811ec79 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/78674fc53d534237bea620882cf183f6_SeqId_28_ 2023-04-09 07:56:00,966 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(849): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fa841uqcc1kq0430ittm5okaasqq009bt6gmhn02ob2ik1gibocnb0nti71ug0o/family1/59db221a30284323ac2b6553e811ec79 into a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/78674fc53d534237bea620882cf183f6_SeqId_28_ - updating store file list. 2023-04-09 07:56:00,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(882): Loaded HFile hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/78674fc53d534237bea620882cf183f6_SeqId_28_ into a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 2023-04-09 07:56:00,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.HStore(855): Successfully loaded hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fa841uqcc1kq0430ittm5okaasqq009bt6gmhn02ob2ik1gibocnb0nti71ug0o/family1/59db221a30284323ac2b6553e811ec79 into a09e145f7e57c7d9f5ff2cc3dd5d6901/family1 (new location: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/family1/78674fc53d534237bea620882cf183f6_SeqId_28_) 2023-04-09 07:56:00,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34907] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/staging/jenkins__testMajorCompactionFromAdmin__fa841uqcc1kq0430ittm5okaasqq009bt6gmhn02ob2ik1gibocnb0nti71ug0o/family1/59db221a30284323ac2b6553e811ec79 2023-04-09 07:56:00,999 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230409 table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:56:01,000 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:56:01,001 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040938e1d3b57c194eea823d2440b39c12b9 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040938e1d3b57c194eea823d2440b39c12b9 2023-04-09 07:56:01,002 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040950058a6cebee416b8d6c70197a9a4965 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023040950058a6cebee416b8d6c70197a9a4965 2023-04-09 07:56:01,003 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304096723b3cfe08940e7ae58636f0f56304d to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202304096723b3cfe08940e7ae58636f0f56304d 2023-04-09 07:56:01,004 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409b34666ee1ee04a4590f6204927086997 to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409b34666ee1ee04a4590f6204927086997 2023-04-09 07:56:01,004 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-04-09 07:56:01,004 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230409' 2023-04-09 07:56:01,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230409 for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:56:01,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-04-09 07:56:01,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-04-09 07:56:01,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del] 2023-04-09 07:56:01,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:56:01,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230409922dfa93eeba42faab6d7ee834d7fdf7_del 2023-04-09 07:56:01,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-09 07:56:01,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del] 2023-04-09 07:56:01,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:56:01,008 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202304096e281ef728f848109b46974e49e56594_del 2023-04-09 07:56:01,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-09 07:56:01,008 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] compactions.PartitionedMobCompactor(367): [hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del] 2023-04-09 07:56:01,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(360): Archiving compacted files. 2023-04-09 07:56:01,010 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del to hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023040971f7dca9c67240cc8243ae51c32d5fad_del 2023-04-09 07:56:01,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-04-09 07:56:01,010 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1681026819944] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-04-09 07:56:01,011 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-04-09 07:56:01,012 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.4710 sec 2023-04-09 07:56:01,030 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:56:01,032 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:56:01,035 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-04-09 07:56:01,036 DEBUG [Listener at localhost/35977] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-04-09 07:56:01,042 INFO [Listener at localhost/35977] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-09 07:56:01,083 INFO [Listener at localhost/35977] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=252 (was 260), OpenFileDescriptor=660 (was 655) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=412 (was 325) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1576 (was 1516) - AvailableMemoryMB LEAK? - 2023-04-09 07:56:01,083 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b944ad to 127.0.0.1:57867 2023-04-09 07:56:01,083 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:01,084 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-04-09 07:56:01,084 INFO [Listener at localhost/35977] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-09 07:56:01,084 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c1e7f16 to 127.0.0.1:57867 2023-04-09 07:56:01,084 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:01,085 DEBUG [Listener at localhost/35977] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-04-09 07:56:01,085 DEBUG [Listener at localhost/35977] util.JVMClusterUtil(257): Found active master hash=710408557, stopped=false 2023-04-09 07:56:01,085 INFO [Listener at localhost/35977] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:56:01,088 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-09 07:56:01,088 INFO [Listener at localhost/35977] procedure2.ProcedureExecutor(629): Stopping 2023-04-09 07:56:01,088 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-09 07:56:01,088 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:56:01,088 DEBUG [Listener at localhost/35977] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3ac6a0e1 to 127.0.0.1:57867 2023-04-09 07:56:01,088 DEBUG [Listener at localhost/35977] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:01,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-09 07:56:01,088 INFO [Listener at localhost/35977] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34907,1681026816245' ***** 2023-04-09 07:56:01,088 INFO [Listener at localhost/35977] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-09 07:56:01,089 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-09 07:56:01,089 INFO [RS:0;jenkins-hbase4:34907] regionserver.HeapMemoryManager(220): Stopping 2023-04-09 07:56:01,089 INFO [RS:0;jenkins-hbase4:34907] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-09 07:56:01,089 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-09 07:56:01,089 INFO [RS:0;jenkins-hbase4:34907] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-09 07:56:01,090 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for fced19ac59ac2562418987229fbaf2d6 2023-04-09 07:56:01,090 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 9be6d4060ae57ce72cac4673686ad32d 2023-04-09 07:56:01,090 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for a09e145f7e57c7d9f5ff2cc3dd5d6901 2023-04-09 07:56:01,090 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for ce3a8bff0d6396ea7236f773f152c0ec 2023-04-09 07:56:01,090 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for aa312bcb3ab129175528fadb46d3a2db 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for fde6fda2ba4dd6622e7e423c1f8b61dd 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for f0431b4fc004fb6549afd7e745a9bb5c 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 60a1e7a3b2b7470f369b184b87cd1dee 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 410e3eed8a4051a7a5cebb43c08ccfed 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 33695bd4d17ece9a6c2338030063f345 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 74a18d993537627879458039595a67bb 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 351ba7e24003baba73ac0a889f713678 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 16deafa9a5d9b83ab649c63be2104462 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 0b0f4838135d08da8ef165e510cc0c2c 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for d4ff53eca0e32f1940dd88a1cd07958a 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 5d73d913ad931f64b26893f58ee65a61 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 6e9e510b2ac2237db799773119e0e62a 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 2c79a4f3105037e03dd30b14fe66503d 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for c669f14846d655d587709ca8bd4b7e00 2023-04-09 07:56:01,091 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:56:01,091 DEBUG [RS:0;jenkins-hbase4:34907] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3b2e629f to 127.0.0.1:57867 2023-04-09 07:56:01,091 DEBUG [RS:0;jenkins-hbase4:34907] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fced19ac59ac2562418987229fbaf2d6, disabling compactions & flushes 2023-04-09 07:56:01,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. after waiting 0 ms 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:56:01,092 INFO [RS:0;jenkins-hbase4:34907] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-09 07:56:01,092 INFO [RS:0;jenkins-hbase4:34907] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-09 07:56:01,092 INFO [RS:0;jenkins-hbase4:34907] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-09 07:56:01,092 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-04-09 07:56:01,092 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-09 07:56:01,092 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-09 07:56:01,092 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-09 07:56:01,093 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-09 07:56:01,093 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-04-09 07:56:01,093 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 0b0f4838135d08da8ef165e510cc0c2c, 1588230740, 16deafa9a5d9b83ab649c63be2104462, 2c79a4f3105037e03dd30b14fe66503d, 33695bd4d17ece9a6c2338030063f345, 351ba7e24003baba73ac0a889f713678, 410e3eed8a4051a7a5cebb43c08ccfed, 5d73d913ad931f64b26893f58ee65a61, 60a1e7a3b2b7470f369b184b87cd1dee, 6e9e510b2ac2237db799773119e0e62a, 74a18d993537627879458039595a67bb, 9be6d4060ae57ce72cac4673686ad32d, a09e145f7e57c7d9f5ff2cc3dd5d6901, aa312bcb3ab129175528fadb46d3a2db, c669f14846d655d587709ca8bd4b7e00, ce3a8bff0d6396ea7236f773f152c0ec, d4ff53eca0e32f1940dd88a1cd07958a, f0431b4fc004fb6549afd7e745a9bb5c, fced19ac59ac2562418987229fbaf2d6, fde6fda2ba4dd6622e7e423c1f8b61dd, fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:56:01,098 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithMonthlyPolicy/fced19ac59ac2562418987229fbaf2d6/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-04-09 07:56:01,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fced19ac59ac2562418987229fbaf2d6: 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1681026830766.fced19ac59ac2562418987229fbaf2d6. 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9be6d4060ae57ce72cac4673686ad32d, disabling compactions & flushes 2023-04-09 07:56:01,099 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. after waiting 0 ms 2023-04-09 07:56:01,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:56:01,107 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-04-09 07:56:01,107 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/9be6d4060ae57ce72cac4673686ad32d/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-09 07:56:01,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:56:01,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9be6d4060ae57ce72cac4673686ad32d: 2023-04-09 07:56:01,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1681026895456.9be6d4060ae57ce72cac4673686ad32d. 2023-04-09 07:56:01,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a09e145f7e57c7d9f5ff2cc3dd5d6901, disabling compactions & flushes 2023-04-09 07:56:01,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:56:01,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:56:01,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. after waiting 1 ms 2023-04-09 07:56:01,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:56:01,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/a09e145f7e57c7d9f5ff2cc3dd5d6901/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:56:01,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a09e145f7e57c7d9f5ff2cc3dd5d6901: 2023-04-09 07:56:01,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1681026945162.a09e145f7e57c7d9f5ff2cc3dd5d6901. 2023-04-09 07:56:01,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ce3a8bff0d6396ea7236f773f152c0ec, disabling compactions & flushes 2023-04-09 07:56:01,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:56:01,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:56:01,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. after waiting 0 ms 2023-04-09 07:56:01,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:56:01,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/ce3a8bff0d6396ea7236f773f152c0ec/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:56:01,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ce3a8bff0d6396ea7236f773f152c0ec: 2023-04-09 07:56:01,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1681026945162.ce3a8bff0d6396ea7236f773f152c0ec. 2023-04-09 07:56:01,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing aa312bcb3ab129175528fadb46d3a2db, disabling compactions & flushes 2023-04-09 07:56:01,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:56:01,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:56:01,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. after waiting 0 ms 2023-04-09 07:56:01,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:56:01,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/aa312bcb3ab129175528fadb46d3a2db/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-04-09 07:56:01,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for aa312bcb3ab129175528fadb46d3a2db: 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1681026877793.aa312bcb3ab129175528fadb46d3a2db. 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fde6fda2ba4dd6622e7e423c1f8b61dd, disabling compactions & flushes 2023-04-09 07:56:01,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. after waiting 0 ms 2023-04-09 07:56:01,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:56:01,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionFromAdmin/fde6fda2ba4dd6622e7e423c1f8b61dd/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fde6fda2ba4dd6622e7e423c1f8b61dd: 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1681026945162.fde6fda2ba4dd6622e7e423c1f8b61dd. 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f0431b4fc004fb6549afd7e745a9bb5c, disabling compactions & flushes 2023-04-09 07:56:01,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. after waiting 0 ms 2023-04-09 07:56:01,129 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:56:01,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/f0431b4fc004fb6549afd7e745a9bb5c/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-04-09 07:56:01,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:56:01,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f0431b4fc004fb6549afd7e745a9bb5c: 2023-04-09 07:56:01,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1681026849725.f0431b4fc004fb6549afd7e745a9bb5c. 2023-04-09 07:56:01,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 60a1e7a3b2b7470f369b184b87cd1dee, disabling compactions & flushes 2023-04-09 07:56:01,133 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:56:01,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:56:01,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. after waiting 0 ms 2023-04-09 07:56:01,133 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:56:01,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/60a1e7a3b2b7470f369b184b87cd1dee/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-09 07:56:01,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:56:01,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 60a1e7a3b2b7470f369b184b87cd1dee: 2023-04-09 07:56:01,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1681026895456.60a1e7a3b2b7470f369b184b87cd1dee. 2023-04-09 07:56:01,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 410e3eed8a4051a7a5cebb43c08ccfed, disabling compactions & flushes 2023-04-09 07:56:01,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:56:01,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:56:01,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. after waiting 0 ms 2023-04-09 07:56:01,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:56:01,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/410e3eed8a4051a7a5cebb43c08ccfed/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,140 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:56:01,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 410e3eed8a4051a7a5cebb43c08ccfed: 2023-04-09 07:56:01,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1681026919825.410e3eed8a4051a7a5cebb43c08ccfed. 2023-04-09 07:56:01,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 33695bd4d17ece9a6c2338030063f345, disabling compactions & flushes 2023-04-09 07:56:01,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:56:01,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:56:01,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. after waiting 0 ms 2023-04-09 07:56:01,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:56:01,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/33695bd4d17ece9a6c2338030063f345/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-04-09 07:56:01,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 33695bd4d17ece9a6c2338030063f345: 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1681026865593.33695bd4d17ece9a6c2338030063f345. 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 74a18d993537627879458039595a67bb, disabling compactions & flushes 2023-04-09 07:56:01,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. after waiting 0 ms 2023-04-09 07:56:01,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:56:01,147 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithMonthlyPolicy/74a18d993537627879458039595a67bb/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-04-09 07:56:01,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 74a18d993537627879458039595a67bb: 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1681026935349.74a18d993537627879458039595a67bb. 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 351ba7e24003baba73ac0a889f713678, disabling compactions & flushes 2023-04-09 07:56:01,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. after waiting 0 ms 2023-04-09 07:56:01,148 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:56:01,148 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-09 07:56:01,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/351ba7e24003baba73ac0a889f713678/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:56:01,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 351ba7e24003baba73ac0a889f713678: 2023-04-09 07:56:01,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1681026919825.351ba7e24003baba73ac0a889f713678. 2023-04-09 07:56:01,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 16deafa9a5d9b83ab649c63be2104462, disabling compactions & flushes 2023-04-09 07:56:01,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:56:01,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:56:01,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. after waiting 0 ms 2023-04-09 07:56:01,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:56:01,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/16deafa9a5d9b83ab649c63be2104462/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-04-09 07:56:01,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:56:01,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 16deafa9a5d9b83ab649c63be2104462: 2023-04-09 07:56:01,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1681026849725.16deafa9a5d9b83ab649c63be2104462. 2023-04-09 07:56:01,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0b0f4838135d08da8ef165e510cc0c2c, disabling compactions & flushes 2023-04-09 07:56:01,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:56:01,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:56:01,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. after waiting 0 ms 2023-04-09 07:56:01,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:56:01,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 0b0f4838135d08da8ef165e510cc0c2c 1/1 column families, dataSize=109 B heapSize=592 B 2023-04-09 07:56:01,293 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 0b0f4838135d08da8ef165e510cc0c2c, 1588230740, 2c79a4f3105037e03dd30b14fe66503d, 5d73d913ad931f64b26893f58ee65a61, 6e9e510b2ac2237db799773119e0e62a, c669f14846d655d587709ca8bd4b7e00, d4ff53eca0e32f1940dd88a1cd07958a, fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:56:01,473 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-04-09 07:56:01,473 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-04-09 07:56:01,494 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 0b0f4838135d08da8ef165e510cc0c2c, 1588230740, 2c79a4f3105037e03dd30b14fe66503d, 5d73d913ad931f64b26893f58ee65a61, 6e9e510b2ac2237db799773119e0e62a, c669f14846d655d587709ca8bd4b7e00, d4ff53eca0e32f1940dd88a1cd07958a, fe2228553ec53030a5c3900b944f37d1 2023-04-09 07:56:01,515 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/info/761f4c81968d4a13b012d06673ef9224 2023-04-09 07:56:01,519 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 761f4c81968d4a13b012d06673ef9224 2023-04-09 07:56:01,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/.tmp/info/b51585ab255742909f363672a48f02f9 2023-04-09 07:56:01,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/.tmp/info/b51585ab255742909f363672a48f02f9 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/info/b51585ab255742909f363672a48f02f9 2023-04-09 07:56:01,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/info/b51585ab255742909f363672a48f02f9, entries=3, sequenceid=7, filesize=4.8 K 2023-04-09 07:56:01,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 0b0f4838135d08da8ef165e510cc0c2c in 418ms, sequenceid=7, compaction requested=false 2023-04-09 07:56:01,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/namespace/0b0f4838135d08da8ef165e510cc0c2c/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-04-09 07:56:01,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0b0f4838135d08da8ef165e510cc0c2c: 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1681026818826.0b0f4838135d08da8ef165e510cc0c2c. 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fe2228553ec53030a5c3900b944f37d1, disabling compactions & flushes 2023-04-09 07:56:01,579 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. after waiting 0 ms 2023-04-09 07:56:01,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:56:01,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMinorCompactionWithWeeklyPolicy/fe2228553ec53030a5c3900b944f37d1/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-04-09 07:56:01,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fe2228553ec53030a5c3900b944f37d1: 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1681026839698.fe2228553ec53030a5c3900b944f37d1. 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d4ff53eca0e32f1940dd88a1cd07958a, disabling compactions & flushes 2023-04-09 07:56:01,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. after waiting 0 ms 2023-04-09 07:56:01,582 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:56:01,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testCompactionWithHFileLink/d4ff53eca0e32f1940dd88a1cd07958a/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-04-09 07:56:01,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d4ff53eca0e32f1940dd88a1cd07958a: 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1681026895456.d4ff53eca0e32f1940dd88a1cd07958a. 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5d73d913ad931f64b26893f58ee65a61, disabling compactions & flushes 2023-04-09 07:56:01,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. after waiting 0 ms 2023-04-09 07:56:01,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:56:01,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5d73d913ad931f64b26893f58ee65a61/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-04-09 07:56:01,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5d73d913ad931f64b26893f58ee65a61: 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1681026820102.5d73d913ad931f64b26893f58ee65a61. 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6e9e510b2ac2237db799773119e0e62a, disabling compactions & flushes 2023-04-09 07:56:01,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. after waiting 0 ms 2023-04-09 07:56:01,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testScannerOnBulkLoadRefHFiles/6e9e510b2ac2237db799773119e0e62a/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-04-09 07:56:01,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6e9e510b2ac2237db799773119e0e62a: 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1681026849725.6e9e510b2ac2237db799773119e0e62a. 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2c79a4f3105037e03dd30b14fe66503d, disabling compactions & flushes 2023-04-09 07:56:01,590 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. after waiting 0 ms 2023-04-09 07:56:01,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:56:01,592 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/default/testMajorCompactionWithWeeklyPolicy/2c79a4f3105037e03dd30b14fe66503d/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-04-09 07:56:01,593 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2c79a4f3105037e03dd30b14fe66503d: 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1681026854719.2c79a4f3105037e03dd30b14fe66503d. 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c669f14846d655d587709ca8bd4b7e00, disabling compactions & flushes 2023-04-09 07:56:01,593 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. after waiting 0 ms 2023-04-09 07:56:01,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:56:01,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/ns/testMinorCompaction/c669f14846d655d587709ca8bd4b7e00/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-04-09 07:56:01,597 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:56:01,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c669f14846d655d587709ca8bd4b7e00: 2023-04-09 07:56:01,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1681026919825.c669f14846d655d587709ca8bd4b7e00. 2023-04-09 07:56:01,694 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-09 07:56:01,894 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-09 07:56:01,927 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/rep_barrier/443bac865c10404cbf29e8243c24ddae 2023-04-09 07:56:01,930 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 443bac865c10404cbf29e8243c24ddae 2023-04-09 07:56:02,095 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-04-09 07:56:02,095 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-04-09 07:56:02,095 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-09 07:56:02,295 DEBUG [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-09 07:56:02,343 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/table/79e114bb939840ad8d9d9536b3c7fad1 2023-04-09 07:56:02,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79e114bb939840ad8d9d9536b3c7fad1 2023-04-09 07:56:02,350 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/info/761f4c81968d4a13b012d06673ef9224 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/info/761f4c81968d4a13b012d06673ef9224 2023-04-09 07:56:02,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 761f4c81968d4a13b012d06673ef9224 2023-04-09 07:56:02,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/info/761f4c81968d4a13b012d06673ef9224, entries=226, sequenceid=130, filesize=37.9 K 2023-04-09 07:56:02,376 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/rep_barrier/443bac865c10404cbf29e8243c24ddae as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/rep_barrier/443bac865c10404cbf29e8243c24ddae 2023-04-09 07:56:02,385 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 443bac865c10404cbf29e8243c24ddae 2023-04-09 07:56:02,385 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/rep_barrier/443bac865c10404cbf29e8243c24ddae, entries=1, sequenceid=130, filesize=5.0 K 2023-04-09 07:56:02,385 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/.tmp/table/79e114bb939840ad8d9d9536b3c7fad1 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/table/79e114bb939840ad8d9d9536b3c7fad1 2023-04-09 07:56:02,388 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 79e114bb939840ad8d9d9536b3c7fad1 2023-04-09 07:56:02,389 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/table/79e114bb939840ad8d9d9536b3c7fad1, entries=27, sequenceid=130, filesize=6.6 K 2023-04-09 07:56:02,391 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1298ms, sequenceid=130, compaction requested=false 2023-04-09 07:56:02,397 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-04-09 07:56:02,397 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-09 07:56:02,397 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-09 07:56:02,397 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-09 07:56:02,398 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-04-09 07:56:02,495 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34907,1681026816245; all regions closed. 2023-04-09 07:56:02,507 DEBUG [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/oldWALs 2023-04-09 07:56:02,507 INFO [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34907%2C1681026816245.meta:.meta(num 1681026818570) 2023-04-09 07:56:02,512 DEBUG [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/oldWALs 2023-04-09 07:56:02,512 INFO [RS:0;jenkins-hbase4:34907] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34907%2C1681026816245:(num 1681026817603) 2023-04-09 07:56:02,512 DEBUG [RS:0;jenkins-hbase4:34907] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:02,512 INFO [RS:0;jenkins-hbase4:34907] regionserver.LeaseManager(133): Closed leases 2023-04-09 07:56:02,513 INFO [RS:0;jenkins-hbase4:34907] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-09 07:56:02,513 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-09 07:56:02,514 INFO [RS:0;jenkins-hbase4:34907] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34907 2023-04-09 07:56:02,521 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-09 07:56:02,521 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34907,1681026816245 2023-04-09 07:56:02,522 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@60514dbe rejected from java.util.concurrent.ThreadPoolExecutor@646ec56c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,522 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-09 07:56:02,522 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34907,1681026816245] 2023-04-09 07:56:02,522 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7ad1ebfa rejected from java.util.concurrent.ThreadPoolExecutor@646ec56c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,524 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34907,1681026816245; numProcessing=1 2023-04-09 07:56:02,526 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34907,1681026816245 already deleted, retry=false 2023-04-09 07:56:02,526 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34907,1681026816245 expired; onlineServers=0 2023-04-09 07:56:02,526 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,44231,1681026814996' ***** 2023-04-09 07:56:02,526 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-04-09 07:56:02,527 DEBUG [M:0;jenkins-hbase4:44231] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b6af356, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-04-09 07:56:02,527 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,44231,1681026814996 2023-04-09 07:56:02,527 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,44231,1681026814996; all regions closed. 2023-04-09 07:56:02,527 DEBUG [M:0;jenkins-hbase4:44231] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-09 07:56:02,527 DEBUG [M:0;jenkins-hbase4:44231] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-04-09 07:56:02,527 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-04-09 07:56:02,527 DEBUG [M:0;jenkins-hbase4:44231] cleaner.HFileCleaner(317): Stopping file delete threads 2023-04-09 07:56:02,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1681026817299] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1681026817299,5,FailOnTimeoutGroup] 2023-04-09 07:56:02,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1681026817305] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1681026817305,5,FailOnTimeoutGroup] 2023-04-09 07:56:02,527 INFO [M:0;jenkins-hbase4:44231] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-04-09 07:56:02,527 INFO [M:0;jenkins-hbase4:44231] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-04-09 07:56:02,528 INFO [M:0;jenkins-hbase4:44231] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-04-09 07:56:02,528 DEBUG [M:0;jenkins-hbase4:44231] master.HMaster(1512): Stopping service threads 2023-04-09 07:56:02,528 INFO [M:0;jenkins-hbase4:44231] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-04-09 07:56:02,529 ERROR [M:0;jenkins-hbase4:44231] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-04-09 07:56:02,529 INFO [M:0;jenkins-hbase4:44231] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-09 07:56:02,530 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-04-09 07:56:02,531 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-04-09 07:56:02,531 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-09 07:56:02,531 DEBUG [M:0;jenkins-hbase4:44231] zookeeper.ZKUtil(398): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-04-09 07:56:02,531 WARN [M:0;jenkins-hbase4:44231] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-04-09 07:56:02,531 INFO [M:0;jenkins-hbase4:44231] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-09 07:56:02,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-09 07:56:02,531 INFO [M:0;jenkins-hbase4:44231] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-04-09 07:56:02,531 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-09 07:56:02,531 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:56:02,531 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:56:02,531 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-09 07:56:02,531 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:56:02,531 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.53 KB heapSize=598.24 KB 2023-04-09 07:56:02,541 INFO [M:0;jenkins-hbase4:44231] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.53 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bbd4e49029ba4b04bba6a473dd82f924 2023-04-09 07:56:02,544 INFO [M:0;jenkins-hbase4:44231] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbd4e49029ba4b04bba6a473dd82f924 2023-04-09 07:56:02,545 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/bbd4e49029ba4b04bba6a473dd82f924 as hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bbd4e49029ba4b04bba6a473dd82f924 2023-04-09 07:56:02,549 INFO [M:0;jenkins-hbase4:44231] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbd4e49029ba4b04bba6a473dd82f924 2023-04-09 07:56:02,549 INFO [M:0;jenkins-hbase4:44231] regionserver.HStore(1080): Added hdfs://localhost:40795/user/jenkins/test-data/dc90fa79-da52-f2c5-828d-4039a8e27b2c/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/bbd4e49029ba4b04bba6a473dd82f924, entries=123, sequenceid=1005, filesize=33.3 K 2023-04-09 07:56:02,550 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegion(2948): Finished flush of dataSize ~512.53 KB/524826, heapSize ~598.23 KB/612584, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1005, compaction requested=false 2023-04-09 07:56:02,550 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-09 07:56:02,550 DEBUG [M:0;jenkins-hbase4:44231] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-09 07:56:02,553 INFO [M:0;jenkins-hbase4:44231] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-04-09 07:56:02,553 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-09 07:56:02,554 INFO [M:0;jenkins-hbase4:44231] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:44231 2023-04-09 07:56:02,558 DEBUG [M:0;jenkins-hbase4:44231] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,44231,1681026814996 already deleted, retry=false 2023-04-09 07:56:02,626 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-09 07:56:02,626 INFO [RS:0;jenkins-hbase4:34907] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34907,1681026816245; zookeeper connection closed. 2023-04-09 07:56:02,626 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@250976a rejected from java.util.concurrent.ThreadPoolExecutor@646ec56c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,626 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): regionserver:34907-0x10167e97a960001, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-09 07:56:02,626 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@77262c06] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@77262c06 2023-04-09 07:56:02,626 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@34381523 rejected from java.util.concurrent.ThreadPoolExecutor@646ec56c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,626 INFO [Listener at localhost/35977] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-04-09 07:56:02,726 INFO [M:0;jenkins-hbase4:44231] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,44231,1681026814996; zookeeper connection closed. 2023-04-09 07:56:02,726 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-09 07:56:02,726 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7eb31653 rejected from java.util.concurrent.ThreadPoolExecutor@5417a87d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,726 DEBUG [Listener at localhost/35977-EventThread] zookeeper.ZKWatcher(600): master:44231-0x10167e97a960000, quorum=127.0.0.1:57867, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-09 07:56:02,726 ERROR [Listener at localhost/35977-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@40c9b5d1 rejected from java.util.concurrent.ThreadPoolExecutor@5417a87d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-09 07:56:02,728 WARN [Listener at localhost/35977] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-09 07:56:02,732 INFO [Listener at localhost/35977] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-09 07:56:02,755 WARN [BP-1755355660-172.31.14.131-1681026811671 heartbeating to localhost/127.0.0.1:40795] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-04-09 07:56:02,755 WARN [BP-1755355660-172.31.14.131-1681026811671 heartbeating to localhost/127.0.0.1:40795] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1755355660-172.31.14.131-1681026811671 (Datanode Uuid 1d6642eb-77e4-4c9a-88ef-513280ce7b16) service to localhost/127.0.0.1:40795 2023-04-09 07:56:02,758 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/dfs/data/data1/current/BP-1755355660-172.31.14.131-1681026811671] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-09 07:56:02,758 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/a1484358-d2d0-79b9-5fa3-b4711fccb934/cluster_4dde5737-39b5-213d-2e47-010bf6b99801/dfs/data/data2/current/BP-1755355660-172.31.14.131-1681026811671] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-09 07:56:02,795 INFO [Listener at localhost/35977] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-09 07:56:02,835 INFO [Listener at localhost/35977] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-04-09 07:56:02,955 INFO [Listener at localhost/35977] hbase.HBaseTestingUtility(1293): Minicluster is down